Intro

Author: Paige (Spencer) Vega, Vanderbilt University, Ken Lau lab
You can also learn more about the Seurat tool by reading their paper:

Data information

The first dataset is 13,019 PBMCs (peripheral blood mononuclear cells). The second is 12,875 PBMCs stimulated with interferon gamma.

Learning goals for section 1

1. Check the quality of the data and filter it.
2. Normalize, scale, and find highly variable genes (HVGs).
3. Dimension reduction.
4. Differential gene expression.

Call on the Seurat function

# load in the Seurat package
library(Seurat)
Loading required package: ggplot2
Loading required package: cowplot

Attaching package: ‘cowplot’

The following object is masked from ‘package:ggplot2’:

    ggsave

Loading required package: Matrix
# set your working directory
directory<-setwd("/Users/paigevega/Desktop/CQS_2018_DiscoveryOrientedDataScience")

Load in your data and create Seurat object

# load in the data
  # 35,635 rows (genes)
  # 13,019 cells (names are barcodes)
pbmc.data <- read.table(paste(directory,"/immune_control_expression_matrix.txt.gz", sep = ""), sep = "\t")
# Create the Seurat object with the raw (non-normalized data).  Keep all genes expressed in >= 5 cells. 
pbmc <- CreateSeuratObject(raw.data = pbmc.data, min.cells = 5, project = "control_PBMC")
# pbmc@raw.data

Filtering

# The number of genes and UMIs (nGene and nUMI) are automatically calculated
# for every object by Seurat. Use violin plots to visualize number genes and number UMIs.
VlnPlot(object = pbmc, features.plot = c("nGene", "nUMI"), nCol = 2, point.size.use = 0.01)

# Note: finding the percent mitochondrial genes for a dataset is a common QC metric, but, this dataset is already pre-processed to remove cells that have mitochondrial gene expression.
# GenePlot is typically used to visualize gene-gene relationships, but can be used to see how any two variables correlate.  We will use it to see how number UMIs correlate to number of genes.
# by the way, what would be "bad" and what would be "good"?
      # bad = low nGene with high nUMI => means there are few types of genes with high expression
      # good = linear correlation nGene to nUMI, with nUMI about 3-5X greater than nGene
GenePlot(object = pbmc, gene1 = "nUMI", gene2 = "nGene")

# Note: if you're trying to remove cells expressing high % mitochondrial genes, plot nUMI to % mito and QC from there.
# Based on the violin plot, filter out cells by choosing low and high thresholds.
# get rid of cells with very few nGenes or way too many nGene
pbmc <- FilterCells(object = pbmc, subset.names = "nGene", low.thresholds = 200, high.thresholds = 1800)
# if you were removing cells with high % mito expression, remove them here with appropriate thresholds.
# pbmc@raw.data

Normalization

# Normalize the data - log normalization is default.
pbmc <- NormalizeData(object = pbmc)
Performing log-normalization
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
# pbmc@data

Find highly variable genes (HVGs)

# Find the top 1,000 most variable genes
  # Calculates average expression and dispersion, using z-scores to determine outliers, which        accounts for the relationship between variability and avg expression.
pbmc <- FindVariableGenes(pbmc)
Calculating gene means
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
Calculating gene variance to mean ratios
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|

# pbmc@var.genes # contains variable genes
# pbmc@hvg.info  # contains results of HVG analysis

Scale data

# removes "uninteresting sources of variation", like technical noise, or cell cycle
# linear regression to predict gene expression. See ?ScaleData for more information.
pbmc <- ScaleData(pbmc)
Scaling data matrix

  |                                                                                      
  |                                                                                |   0%
  |                                                                                      
  |======                                                                          |   7%
  |                                                                                      
  |===========                                                                     |  14%
  |                                                                                      
  |=================                                                               |  21%
  |                                                                                      
  |=======================                                                         |  29%
  |                                                                                      
  |=============================                                                   |  36%
  |                                                                                      
  |==================================                                              |  43%
  |                                                                                      
  |========================================                                        |  50%
  |                                                                                      
  |==============================================                                  |  57%
  |                                                                                      
  |===================================================                             |  64%
  |                                                                                      
  |=========================================================                       |  71%
  |                                                                                      
  |===============================================================                 |  79%
  |                                                                                      
  |=====================================================================           |  86%
  |                                                                                      
  |==========================================================================      |  93%
  |                                                                                      
  |================================================================================| 100%
# pbmc@scale.data

Dimensionality reduction (PCA)

# Run PCA using the HVGs
pbmc <- RunPCA(object = pbmc, pc.genes = pbmc@var.genes, pcs.compute = 30, do.print = FALSE)
# pbmc@dr$pca

Choose PCs

# Principle components (PCs) capture the variability in your dataset.  Although we specified 30 PCs, we want to use only the PCs that capture variability and discard the rest for downstream analyses.  You look for saturation (flatline) in the relationship between the number of principle components and the percentage of the variance explained.
# 3 ways to look at PCs...
# 1. Elbow plot of principal components
PCElbowPlot(object = pbmc, num.pc = 30)

# 2. Heatmap of individual PCs
PCHeatmap(object = pbmc, pc.use = 1:15, cells.use = 500, do.balanced = TRUE, label.column = FALSE, use.full = FALSE)

PCHeatmap(object = pbmc, pc.use = 16:30, cells.use = 500, do.balanced = TRUE, label.column = FALSE, use.full = FALSE)

# 3. Jackstraw package to determine statistically significant principal components
pbmc <- JackStraw(object = pbmc, num.pc = 30, num.replicate = 100, display.progress = TRUE)

  |                                                                                      
  |                                                                                |   0%
  |                                                                                      
  |=                                                                               |   1%
  |                                                                                      
  |==                                                                              |   2%
  |                                                                                      
  |==                                                                              |   3%
  |                                                                                      
  |===                                                                             |   4%
  |                                                                                      
  |====                                                                            |   5%
  |                                                                                      
  |=====                                                                           |   6%
  |                                                                                      
  |======                                                                          |   7%
  |                                                                                      
  |======                                                                          |   8%
  |                                                                                      
  |=======                                                                         |   9%
  |                                                                                      
  |========                                                                        |  10%
  |                                                                                      
  |=========                                                                       |  11%
  |                                                                                      
  |==========                                                                      |  12%
  |                                                                                      
  |==========                                                                      |  13%
  |                                                                                      
  |===========                                                                     |  14%
  |                                                                                      
  |============                                                                    |  15%
  |                                                                                      
  |=============                                                                   |  16%
  |                                                                                      
  |==============                                                                  |  17%
  |                                                                                      
  |==============                                                                  |  18%
  |                                                                                      
  |===============                                                                 |  19%
  |                                                                                      
  |================                                                                |  20%
  |                                                                                      
  |=================                                                               |  21%
  |                                                                                      
  |==================                                                              |  22%
  |                                                                                      
  |==================                                                              |  23%
  |                                                                                      
  |===================                                                             |  24%
  |                                                                                      
  |====================                                                            |  25%
  |                                                                                      
  |=====================                                                           |  26%
  |                                                                                      
  |======================                                                          |  27%
  |                                                                                      
  |======================                                                          |  28%
  |                                                                                      
  |=======================                                                         |  29%
  |                                                                                      
  |========================                                                        |  30%
  |                                                                                      
  |=========================                                                       |  31%
  |                                                                                      
  |==========================                                                      |  32%
  |                                                                                      
  |==========================                                                      |  33%
  |                                                                                      
  |===========================                                                     |  34%
  |                                                                                      
  |============================                                                    |  35%
  |                                                                                      
  |=============================                                                   |  36%
  |                                                                                      
  |==============================                                                  |  37%
  |                                                                                      
  |==============================                                                  |  38%
  |                                                                                      
  |===============================                                                 |  39%
  |                                                                                      
  |================================                                                |  40%
  |                                                                                      
  |=================================                                               |  41%
  |                                                                                      
  |==================================                                              |  42%
  |                                                                                      
  |==================================                                              |  43%
  |                                                                                      
  |===================================                                             |  44%
  |                                                                                      
  |====================================                                            |  45%
  |                                                                                      
  |=====================================                                           |  46%
  |                                                                                      
  |======================================                                          |  47%
  |                                                                                      
  |======================================                                          |  48%
  |                                                                                      
  |=======================================                                         |  49%
  |                                                                                      
  |========================================                                        |  50%
  |                                                                                      
  |=========================================                                       |  51%
  |                                                                                      
  |==========================================                                      |  52%
  |                                                                                      
  |==========================================                                      |  53%
  |                                                                                      
  |===========================================                                     |  54%
  |                                                                                      
  |============================================                                    |  55%
  |                                                                                      
  |=============================================                                   |  56%
  |                                                                                      
  |==============================================                                  |  57%
  |                                                                                      
  |==============================================                                  |  58%
  |                                                                                      
  |===============================================                                 |  59%
  |                                                                                      
  |================================================                                |  60%
  |                                                                                      
  |=================================================                               |  61%
  |                                                                                      
  |==================================================                              |  62%
  |                                                                                      
  |==================================================                              |  63%
  |                                                                                      
  |===================================================                             |  64%
  |                                                                                      
  |====================================================                            |  65%
  |                                                                                      
  |=====================================================                           |  66%
  |                                                                                      
  |======================================================                          |  67%
  |                                                                                      
  |======================================================                          |  68%
  |                                                                                      
  |=======================================================                         |  69%
  |                                                                                      
  |========================================================                        |  70%
  |                                                                                      
  |=========================================================                       |  71%
  |                                                                                      
  |==========================================================                      |  72%
  |                                                                                      
  |==========================================================                      |  73%
  |                                                                                      
  |===========================================================                     |  74%
  |                                                                                      
  |============================================================                    |  75%
  |                                                                                      
  |=============================================================                   |  76%
  |                                                                                      
  |==============================================================                  |  77%
  |                                                                                      
  |==============================================================                  |  78%
  |                                                                                      
  |===============================================================                 |  79%
  |                                                                                      
  |================================================================                |  80%
  |                                                                                      
  |=================================================================               |  81%
  |                                                                                      
  |==================================================================              |  82%
  |                                                                                      
  |==================================================================              |  83%
  |                                                                                      
  |===================================================================             |  84%
  |                                                                                      
  |====================================================================            |  85%
  |                                                                                      
  |=====================================================================           |  86%
  |                                                                                      
  |======================================================================          |  87%
  |                                                                                      
  |======================================================================          |  88%
  |                                                                                      
  |=======================================================================         |  89%
  |                                                                                      
  |========================================================================        |  90%
  |                                                                                      
  |=========================================================================       |  91%
  |                                                                                      
  |==========================================================================      |  92%
  |                                                                                      
  |==========================================================================      |  93%
  |                                                                                      
  |===========================================================================     |  94%
  |                                                                                      
  |============================================================================    |  95%
  |                                                                                      
  |=============================================================================   |  96%
  |                                                                                      
  |==============================================================================  |  97%
  |                                                                                      
  |==============================================================================  |  98%
  |                                                                                      
  |=============================================================================== |  99%
  |                                                                                      
  |================================================================================| 100%
Time Elapsed:  12.1679507493973 mins 
JackStrawPlot(object = pbmc, PCs = 1:30)
An object of class seurat in project control_PBMC 
 13537 genes across 12927 samples.

Use PCs to cluster and visualize clustering with t-SNE

# Clustering using Seurat's method. See more info in ?FindClusters.
  # specify the number of PCs using dims.use
pbmc <- FindClusters(object = pbmc, reduction.type = "pca", dims.use = 1:20, resolution = 0.6,
                     print.output = 0, save.SNN = TRUE)
# pbmc@ident
# Run dimension reduction (t-SNE) to visualize the clustering
pbmc <- Seurat::RunTSNE(object = pbmc, dims.use = 1:20, do.fast = TRUE)
TSNEPlot(object = pbmc)

Differential gene expression - one cluster at a time

# cluster 1 vs all other clusters pooled together
cluster1.markers <- FindMarkers(object = pbmc, ident.1 = 1, min.pct = 0.25)

   |                                                  | 0 % ~calculating  
   |+                                                 | 1 % ~02m 02s      
   |+                                                 | 2 % ~02m 05s      
   |++                                                | 3 % ~02m 03s      
   |++                                                | 4 % ~02m 02s      
   |+++                                               | 5 % ~01m 59s      
   |+++                                               | 6 % ~01m 57s      
   |++++                                              | 7 % ~01m 56s      
   |++++                                              | 8 % ~01m 57s      
   |+++++                                             | 9 % ~01m 55s      
   |+++++                                             | 10% ~01m 53s      
   |++++++                                            | 11% ~01m 52s      
   |++++++                                            | 12% ~01m 53s      
   |+++++++                                           | 13% ~01m 51s      
   |+++++++                                           | 14% ~01m 50s      
   |++++++++                                          | 15% ~01m 49s      
   |++++++++                                          | 16% ~01m 48s      
   |+++++++++                                         | 17% ~01m 48s      
   |+++++++++                                         | 18% ~01m 50s      
   |++++++++++                                        | 19% ~01m 48s      
   |++++++++++                                        | 20% ~01m 47s      
   |+++++++++++                                       | 21% ~01m 45s      
   |+++++++++++                                       | 22% ~01m 46s      
   |++++++++++++                                      | 23% ~01m 44s      
   |++++++++++++                                      | 24% ~01m 43s      
   |+++++++++++++                                     | 25% ~01m 42s      
   |+++++++++++++                                     | 26% ~01m 41s      
   |++++++++++++++                                    | 27% ~01m 40s      
   |++++++++++++++                                    | 28% ~01m 38s      
   |+++++++++++++++                                   | 29% ~01m 36s      
   |+++++++++++++++                                   | 30% ~01m 36s      
   |++++++++++++++++                                  | 31% ~01m 34s      
   |++++++++++++++++                                  | 32% ~01m 35s      
   |+++++++++++++++++                                 | 33% ~01m 33s      
   |+++++++++++++++++                                | 34% ~01m 31s      
   |++++++++++++++++++                                | 35% ~01m 29s      
   |++++++++++++++++++                                | 36% ~01m 28s      
   |+++++++++++++++++++                               | 37% ~01m 26s      
   |+++++++++++++++++++                               | 38% ~01m 25s      
   |++++++++++++++++++++                              | 39% ~01m 23s      
   |++++++++++++++++++++                              | 40% ~01m 22s      
   |+++++++++++++++++++++                             | 41% ~01m 20s      
   |+++++++++++++++++++++                             | 42% ~01m 19s      
   |++++++++++++++++++++++                            | 43% ~01m 18s      
   |++++++++++++++++++++++                            | 44% ~01m 16s      
   |+++++++++++++++++++++++                           | 45% ~01m 15s      
   |+++++++++++++++++++++++                           | 46% ~01m 13s      
   |++++++++++++++++++++++++                          | 47% ~01m 12s      
   |++++++++++++++++++++++++                          | 48% ~01m 11s      
   |+++++++++++++++++++++++++                         | 49% ~01m 09s      
   |+++++++++++++++++++++++++                         | 50% ~01m 08s      
   |++++++++++++++++++++++++++                        | 51% ~01m 06s      
   |++++++++++++++++++++++++++                        | 52% ~01m 05s      
   |+++++++++++++++++++++++++++                       | 53% ~01m 03s      
   |+++++++++++++++++++++++++++                       | 54% ~01m 02s      
   |++++++++++++++++++++++++++++                      | 55% ~01m 01s      
   |++++++++++++++++++++++++++++                     | 56% ~59s          
   |+++++++++++++++++++++++++++++                     | 57% ~58s          
   |+++++++++++++++++++++++++++++                     | 58% ~56s          
   |++++++++++++++++++++++++++++++                    | 59% ~55s          
   |++++++++++++++++++++++++++++++                    | 60% ~54s          
   |+++++++++++++++++++++++++++++++                   | 61% ~52s          
   |+++++++++++++++++++++++++++++++                   | 62% ~51s          
   |++++++++++++++++++++++++++++++++                  | 63% ~49s          
   |++++++++++++++++++++++++++++++++                  | 64% ~49s          
   |+++++++++++++++++++++++++++++++++                 | 65% ~47s          
   |+++++++++++++++++++++++++++++++++                 | 66% ~46s          
   |++++++++++++++++++++++++++++++++++                | 67% ~44s          
   |++++++++++++++++++++++++++++++++++               | 68% ~43s          
   |+++++++++++++++++++++++++++++++++++               | 69% ~42s          
   |+++++++++++++++++++++++++++++++++++               | 70% ~40s          
   |++++++++++++++++++++++++++++++++++++              | 71% ~39s          
   |++++++++++++++++++++++++++++++++++++              | 72% ~38s          
   |+++++++++++++++++++++++++++++++++++++             | 73% ~36s          
   |+++++++++++++++++++++++++++++++++++++             | 74% ~35s          
   |++++++++++++++++++++++++++++++++++++++            | 75% ~33s          
   |++++++++++++++++++++++++++++++++++++++            | 76% ~32s          
   |+++++++++++++++++++++++++++++++++++++++           | 77% ~31s          
   |+++++++++++++++++++++++++++++++++++++++          | 78% ~29s          
   |++++++++++++++++++++++++++++++++++++++++          | 79% ~28s          
   |++++++++++++++++++++++++++++++++++++++++         | 80% ~27s          
   |+++++++++++++++++++++++++++++++++++++++++         | 81% ~25s          
   |+++++++++++++++++++++++++++++++++++++++++         | 82% ~24s          
   |++++++++++++++++++++++++++++++++++++++++++        | 83% ~23s          
   |++++++++++++++++++++++++++++++++++++++++++        | 84% ~21s          
   |+++++++++++++++++++++++++++++++++++++++++++       | 85% ~20s          
   |+++++++++++++++++++++++++++++++++++++++++++       | 86% ~19s          
   |++++++++++++++++++++++++++++++++++++++++++++      | 87% ~17s          
   |++++++++++++++++++++++++++++++++++++++++++++      | 88% ~16s          
   |+++++++++++++++++++++++++++++++++++++++++++++     | 89% ~15s          
   |+++++++++++++++++++++++++++++++++++++++++++++    | 90% ~13s          
   |++++++++++++++++++++++++++++++++++++++++++++++    | 91% ~12s          
   |++++++++++++++++++++++++++++++++++++++++++++++   | 92% ~11s          
   |+++++++++++++++++++++++++++++++++++++++++++++++   | 93% ~09s          
   |+++++++++++++++++++++++++++++++++++++++++++++++   | 94% ~08s          
   |++++++++++++++++++++++++++++++++++++++++++++++++  | 95% ~07s          
   |++++++++++++++++++++++++++++++++++++++++++++++++  | 96% ~05s          
   |+++++++++++++++++++++++++++++++++++++++++++++++++ | 97% ~04s          
   |+++++++++++++++++++++++++++++++++++++++++++++++++ | 98% ~03s          
   |++++++++++++++++++++++++++++++++++++++++++++++++++| 99% ~01s          
   |++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed = 02m 13s
print(x = head(x = cluster1.markers, n =5))
       p_val avg_logFC pct.1 pct.2 p_val_adj
S100A8     0  2.965944 0.852 0.050         0
CCL2       0  2.926204 0.468 0.038         0
IL8        0  2.769285 0.880 0.091         0
S100A9     0  2.730468 0.762 0.043         0
FTL        0  2.405877 1.000 0.872         0
# cluster 1 vs cluster 5
cluster1.markers.5 <- FindMarkers(object = pbmc, ident.1 = 1, ident.2 = 5, min.pct = 0.25)

   |                                                  | 0 % ~calculating  
   |+                                                 | 1 % ~18s          
   |++                                                | 2 % ~17s          
   |++                                                | 3 % ~17s          
   |+++                                               | 5 % ~17s          
   |+++                                               | 6 % ~16s          
   |++++                                              | 7 % ~16s          
   |+++++                                             | 8 % ~16s          
   |+++++                                             | 9 % ~15s          
   |++++++                                            | 10% ~15s          
   |++++++                                            | 11% ~15s          
   |+++++++                                           | 13% ~14s          
   |+++++++                                           | 14% ~14s          
   |++++++++                                          | 15% ~14s          
   |+++++++++                                         | 16% ~14s          
   |+++++++++                                         | 17% ~13s          
   |++++++++++                                        | 18% ~13s          
   |++++++++++                                        | 20% ~13s          
   |+++++++++++                                       | 21% ~13s          
   |+++++++++++                                       | 22% ~12s          
   |++++++++++++                                      | 23% ~12s          
   |+++++++++++++                                     | 24% ~12s          
   |+++++++++++++                                     | 25% ~12s          
   |++++++++++++++                                    | 26% ~12s          
   |++++++++++++++                                    | 28% ~11s          
   |+++++++++++++++                                   | 29% ~11s          
   |+++++++++++++++                                   | 30% ~11s          
   |++++++++++++++++                                  | 31% ~11s          
   |+++++++++++++++++                                 | 32% ~11s          
   |+++++++++++++++++                                 | 33% ~11s          
   |++++++++++++++++++                                | 34% ~11s          
   |++++++++++++++++++                                | 36% ~11s          
   |+++++++++++++++++++                               | 37% ~11s          
   |+++++++++++++++++++                               | 38% ~11s          
   |++++++++++++++++++++                              | 39% ~11s          
   |+++++++++++++++++++++                             | 40% ~10s          
   |+++++++++++++++++++++                             | 41% ~10s          
   |++++++++++++++++++++++                            | 43% ~10s          
   |++++++++++++++++++++++                            | 44% ~10s          
   |+++++++++++++++++++++++                           | 45% ~10s          
   |+++++++++++++++++++++++                           | 46% ~09s          
   |++++++++++++++++++++++++                          | 47% ~09s          
   |+++++++++++++++++++++++++                         | 48% ~09s          
   |+++++++++++++++++++++++++                         | 49% ~09s          
   |++++++++++++++++++++++++++                        | 51% ~09s          
   |++++++++++++++++++++++++++                        | 52% ~08s          
   |+++++++++++++++++++++++++++                       | 53% ~08s          
   |++++++++++++++++++++++++++++                      | 54% ~08s          
   |++++++++++++++++++++++++++++                      | 55% ~08s          
   |+++++++++++++++++++++++++++++                     | 56% ~08s          
   |+++++++++++++++++++++++++++++                     | 57% ~07s          
   |++++++++++++++++++++++++++++++                    | 59% ~07s          
   |++++++++++++++++++++++++++++++                    | 60% ~07s          
   |+++++++++++++++++++++++++++++++                   | 61% ~07s          
   |++++++++++++++++++++++++++++++++                  | 62% ~07s          
   |++++++++++++++++++++++++++++++++                  | 63% ~06s          
   |+++++++++++++++++++++++++++++++++                 | 64% ~06s          
   |+++++++++++++++++++++++++++++++++                 | 66% ~06s          
   |++++++++++++++++++++++++++++++++++                | 67% ~06s          
   |++++++++++++++++++++++++++++++++++                | 68% ~06s          
   |+++++++++++++++++++++++++++++++++++               | 69% ~05s          
   |++++++++++++++++++++++++++++++++++++              | 70% ~05s          
   |++++++++++++++++++++++++++++++++++++              | 71% ~05s          
   |+++++++++++++++++++++++++++++++++++++             | 72% ~05s          
   |+++++++++++++++++++++++++++++++++++++             | 74% ~05s          
   |++++++++++++++++++++++++++++++++++++++            | 75% ~04s          
   |++++++++++++++++++++++++++++++++++++++            | 76% ~04s          
   |+++++++++++++++++++++++++++++++++++++++           | 77% ~04s          
   |++++++++++++++++++++++++++++++++++++++++          | 78% ~04s          
   |++++++++++++++++++++++++++++++++++++++++          | 79% ~04s          
   |+++++++++++++++++++++++++++++++++++++++++         | 80% ~03s          
   |+++++++++++++++++++++++++++++++++++++++++         | 82% ~03s          
   |++++++++++++++++++++++++++++++++++++++++++        | 83% ~03s          
   |++++++++++++++++++++++++++++++++++++++++++        | 84% ~03s          
   |+++++++++++++++++++++++++++++++++++++++++++       | 85% ~03s          
   |++++++++++++++++++++++++++++++++++++++++++++      | 86% ~02s          
   |++++++++++++++++++++++++++++++++++++++++++++      | 87% ~02s          
   |+++++++++++++++++++++++++++++++++++++++++++++     | 89% ~02s          
   |+++++++++++++++++++++++++++++++++++++++++++++     | 90% ~02s          
   |++++++++++++++++++++++++++++++++++++++++++++++    | 91% ~02s          
   |++++++++++++++++++++++++++++++++++++++++++++++    | 92% ~01s          
   |+++++++++++++++++++++++++++++++++++++++++++++++   | 93% ~01s          
   |++++++++++++++++++++++++++++++++++++++++++++++++  | 94% ~01s          
   |++++++++++++++++++++++++++++++++++++++++++++++++  | 95% ~01s          
   |+++++++++++++++++++++++++++++++++++++++++++++++++ | 97% ~01s          
   |+++++++++++++++++++++++++++++++++++++++++++++++++ | 98% ~00s          
   |++++++++++++++++++++++++++++++++++++++++++++++++++| 99% ~00s          
   |++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed = 18s
print(x = head(x = cluster1.markers.5, n =5))
                 p_val avg_logFC pct.1 pct.2     p_val_adj
VMO1     1.981317e-239 -2.590454 0.100 0.620 2.682109e-235
TNFSF10  4.458680e-222 -1.693412 0.023 0.434 6.035714e-218
FCGR3A   7.791344e-195 -1.641433 0.408 0.841 1.054714e-190
TNFSF13B 2.400820e-180 -1.271797 0.051 0.459 3.249990e-176
FAM26F   4.024863e-177 -1.594279 0.066 0.482 5.448457e-173

Differential gene expression - iterate through all clusters

# Find markers for every cluster compared to all remaining cells, report only positive ones
pbmc.markers <- FindAllMarkers(object = pbmc, only.pos = TRUE, min.pct = 0.25, thresh.use = 0.25)

   |                                                  | 0 % ~calculating  
   |+                                                 | 1 % ~02m 41s      
   |++                                                | 2 % ~02m 09s      
   |++                                                | 4 % ~01m 54s      
   |+++                                               | 5 % ~01m 45s      
   |+++                                               | 6 % ~01m 41s      
   |++++                                              | 7 % ~01m 37s      
   |+++++                                             | 8 % ~01m 39s      
   |+++++                                             | 9 % ~02m 08s      
   |++++++                                            | 11% ~02m 00s      
   |++++++                                            | 12% ~01m 53s      
   |+++++++                                           | 13% ~01m 47s      
   |++++++++                                          | 14% ~01m 42s      
   |++++++++                                          | 15% ~01m 38s      
   |+++++++++                                         | 16% ~01m 36s      
   |+++++++++                                         | 18% ~01m 33s      
   |++++++++++                                        | 19% ~01m 31s      
   |++++++++++                                        | 20% ~01m 28s      
   |+++++++++++                                       | 21% ~01m 26s      
   |++++++++++++                                      | 22% ~01m 24s      
   |++++++++++++                                      | 24% ~01m 22s      
   |+++++++++++++                                     | 25% ~01m 20s      
   |+++++++++++++                                     | 26% ~01m 18s      
   |++++++++++++++                                    | 27% ~01m 16s      
   |+++++++++++++++                                   | 28% ~01m 14s      
   |+++++++++++++++                                   | 29% ~01m 13s      
   |++++++++++++++++                                  | 31% ~01m 11s      
   |++++++++++++++++                                  | 32% ~01m 10s      
   |+++++++++++++++++                                 | 33% ~01m 08s      
   |++++++++++++++++++                                | 34% ~01m 06s      
   |++++++++++++++++++                                | 35% ~01m 05s      
   |+++++++++++++++++++                               | 36% ~01m 03s      
   |+++++++++++++++++++                               | 38% ~01m 02s      
   |++++++++++++++++++++                              | 39% ~01m 01s      
   |++++++++++++++++++++                              | 40% ~59s          
   |+++++++++++++++++++++                             | 41% ~58s          
   |++++++++++++++++++++++                            | 42% ~57s          
   |++++++++++++++++++++++                            | 44% ~55s          
   |+++++++++++++++++++++++                           | 45% ~54s          
   |+++++++++++++++++++++++                           | 46% ~53s          
   |++++++++++++++++++++++++                          | 47% ~51s          
   |+++++++++++++++++++++++++                         | 48% ~50s          
   |+++++++++++++++++++++++++                         | 49% ~49s          
   |++++++++++++++++++++++++++                        | 51% ~48s          
   |++++++++++++++++++++++++++                        | 52% ~46s          
   |+++++++++++++++++++++++++++                       | 53% ~45s          
   |++++++++++++++++++++++++++++                      | 54% ~44s          
   |++++++++++++++++++++++++++++                      | 55% ~43s          
   |+++++++++++++++++++++++++++++                     | 56% ~42s          
   |+++++++++++++++++++++++++++++                     | 58% ~40s          
   |++++++++++++++++++++++++++++++                    | 59% ~39s          
   |++++++++++++++++++++++++++++++                    | 60% ~39s          
   |+++++++++++++++++++++++++++++++                   | 61% ~38s          
   |++++++++++++++++++++++++++++++++                  | 62% ~36s          
   |++++++++++++++++++++++++++++++++                  | 64% ~35s          
   |+++++++++++++++++++++++++++++++++                 | 65% ~34s          
   |+++++++++++++++++++++++++++++++++                 | 66% ~33s          
   |++++++++++++++++++++++++++++++++++                | 67% ~32s          
   |+++++++++++++++++++++++++++++++++++               | 68% ~31s          
   |+++++++++++++++++++++++++++++++++++               | 69% ~29s          
   |++++++++++++++++++++++++++++++++++++              | 71% ~28s          
   |++++++++++++++++++++++++++++++++++++              | 72% ~27s          
   |+++++++++++++++++++++++++++++++++++++             | 73% ~26s          
   |++++++++++++++++++++++++++++++++++++++            | 74% ~25s          
   |++++++++++++++++++++++++++++++++++++++            | 75% ~24s          
   |+++++++++++++++++++++++++++++++++++++++           | 76% ~23s          
   |+++++++++++++++++++++++++++++++++++++++           | 78% ~21s          
   |++++++++++++++++++++++++++++++++++++++++          | 79% ~20s          
   |++++++++++++++++++++++++++++++++++++++++         | 80% ~19s          
   |+++++++++++++++++++++++++++++++++++++++++         | 81% ~18s          
   |++++++++++++++++++++++++++++++++++++++++++        | 82% ~17s          
   |++++++++++++++++++++++++++++++++++++++++++        | 84% ~16s          
   |+++++++++++++++++++++++++++++++++++++++++++       | 85% ~14s          
   |+++++++++++++++++++++++++++++++++++++++++++       | 86% ~13s          
   |++++++++++++++++++++++++++++++++++++++++++++      | 87% ~12s          
   |+++++++++++++++++++++++++++++++++++++++++++++     | 88% ~11s          
   |+++++++++++++++++++++++++++++++++++++++++++++     | 89% ~10s          
   |++++++++++++++++++++++++++++++++++++++++++++++    | 91% ~09s          
   |++++++++++++++++++++++++++++++++++++++++++++++    | 92% ~08s          
   |+++++++++++++++++++++++++++++++++++++++++++++++   | 93% ~07s          
   |++++++++++++++++++++++++++++++++++++++++++++++++  | 94% ~05s          
   |++++++++++++++++++++++++++++++++++++++++++++++++  | 95% ~04s          
   |+++++++++++++++++++++++++++++++++++++++++++++++++ | 96% ~03s          
   |+++++++++++++++++++++++++++++++++++++++++++++++++ | 98% ~02s          
   |++++++++++++++++++++++++++++++++++++++++++++++++++| 99% ~01s          
   |++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed = 01m 34s

   |                                                  | 0 % ~calculating  
   |+                                                 | 1 % ~02m 25s      
   |+                                                 | 2 % ~02m 26s      
   |++                                                | 3 % ~02m 32s      
   |++                                                | 4 % ~02m 28s      
   |+++                                               | 5 % ~02m 23s      
   |+++                                               | 6 % ~02m 17s      
   |++++                                              | 7 % ~02m 13s      
   |++++                                              | 8 % ~02m 09s      
   |+++++                                             | 9 % ~02m 06s      
   |+++++                                             | 10% ~02m 04s      
   |++++++                                            | 11% ~02m 03s      
   |++++++                                            | 12% ~02m 02s      
   |+++++++                                           | 13% ~01m 59s      
   |+++++++                                           | 14% ~01m 60s      
   |++++++++                                          | 15% ~01m 58s      
   |++++++++                                          | 16% ~01m 57s      
   |+++++++++                                         | 17% ~02m 04s      
   |+++++++++                                         | 18% ~02m 01s      
   |++++++++++                                        | 19% ~01m 58s      
   |++++++++++                                        | 20% ~01m 56s      
   |+++++++++++                                       | 21% ~01m 54s      
   |+++++++++++                                       | 22% ~01m 52s      
   |++++++++++++                                      | 23% ~01m 50s      
   |++++++++++++                                      | 24% ~01m 49s      
   |+++++++++++++                                     | 25% ~01m 47s      
   |+++++++++++++                                     | 26% ~01m 45s      
   |++++++++++++++                                    | 27% ~01m 43s      
   |++++++++++++++                                    | 28% ~01m 41s      
   |+++++++++++++++                                   | 29% ~01m 39s      
   |+++++++++++++++                                   | 30% ~01m 37s      
   |++++++++++++++++                                  | 31% ~01m 36s      
   |++++++++++++++++                                  | 32% ~01m 34s      
   |+++++++++++++++++                                 | 33% ~01m 33s      
   |+++++++++++++++++                                | 34% ~01m 31s      
   |++++++++++++++++++                                | 35% ~01m 29s      
   |++++++++++++++++++                                | 36% ~01m 28s      
   |+++++++++++++++++++                               | 37% ~01m 26s      
   |+++++++++++++++++++                               | 38% ~01m 25s      
   |++++++++++++++++++++                              | 39% ~01m 23s      
   |++++++++++++++++++++                              | 40% ~01m 22s      
   |+++++++++++++++++++++                             | 41% ~01m 20s      
   |+++++++++++++++++++++                             | 42% ~01m 19s      
   |++++++++++++++++++++++                            | 43% ~01m 18s      
   |++++++++++++++++++++++                            | 44% ~01m 16s      
   |+++++++++++++++++++++++                           | 45% ~01m 16s      
   |+++++++++++++++++++++++                           | 46% ~01m 15s      
   |++++++++++++++++++++++++                          | 47% ~01m 13s      
   |++++++++++++++++++++++++                          | 48% ~01m 12s      
   |+++++++++++++++++++++++++                         | 49% ~01m 11s      
   |+++++++++++++++++++++++++                         | 50% ~01m 09s      
   |++++++++++++++++++++++++++                        | 51% ~01m 08s      
   |++++++++++++++++++++++++++                        | 52% ~01m 06s      
   |+++++++++++++++++++++++++++                       | 53% ~01m 04s      
   |+++++++++++++++++++++++++++                       | 54% ~01m 03s      
   |++++++++++++++++++++++++++++                      | 55% ~01m 02s      
   |++++++++++++++++++++++++++++                     | 56% ~01m 00s      
   |+++++++++++++++++++++++++++++                     | 57% ~59s          
   |+++++++++++++++++++++++++++++                     | 58% ~57s          
   |++++++++++++++++++++++++++++++                    | 59% ~56s          
   |++++++++++++++++++++++++++++++                    | 60% ~55s          
   |+++++++++++++++++++++++++++++++                   | 61% ~53s          
   |+++++++++++++++++++++++++++++++                   | 62% ~52s          
   |++++++++++++++++++++++++++++++++                  | 63% ~50s          
   |++++++++++++++++++++++++++++++++                  | 64% ~49s          
   |+++++++++++++++++++++++++++++++++                 | 65% ~48s          
   |+++++++++++++++++++++++++++++++++                 | 66% ~47s          
   |++++++++++++++++++++++++++++++++++                | 67% ~45s          
   |++++++++++++++++++++++++++++++++++               | 68% ~44s          
   |+++++++++++++++++++++++++++++++++++               | 69% ~42s          
   |+++++++++++++++++++++++++++++++++++               | 70% ~41s          
   |++++++++++++++++++++++++++++++++++++              | 71% ~40s          
   |++++++++++++++++++++++++++++++++++++              | 72% ~39s          
   |+++++++++++++++++++++++++++++++++++++             | 73% ~37s          
   |+++++++++++++++++++++++++++++++++++++             | 74% ~36s          
   |++++++++++++++++++++++++++++++++++++++            | 75% ~34s          
   |++++++++++++++++++++++++++++++++++++++            | 76% ~33s          
   |+++++++++++++++++++++++++++++++++++++++           | 77% ~32s          
   |+++++++++++++++++++++++++++++++++++++++          | 78% ~30s          
   |++++++++++++++++++++++++++++++++++++++++          | 79% ~29s          
   |++++++++++++++++++++++++++++++++++++++++         | 80% ~27s          
   |+++++++++++++++++++++++++++++++++++++++++         | 81% ~26s          
   |+++++++++++++++++++++++++++++++++++++++++         | 82% ~25s          
   |++++++++++++++++++++++++++++++++++++++++++        | 83% ~23s          
   |++++++++++++++++++++++++++++++++++++++++++        | 84% ~22s          
   |+++++++++++++++++++++++++++++++++++++++++++       | 85% ~21s          
   |+++++++++++++++++++++++++++++++++++++++++++       | 86% ~19s          
   |++++++++++++++++++++++++++++++++++++++++++++      | 87% ~18s          
   |++++++++++++++++++++++++++++++++++++++++++++      | 88% ~16s          
   |+++++++++++++++++++++++++++++++++++++++++++++     | 89% ~15s          
   |+++++++++++++++++++++++++++++++++++++++++++++    | 90% ~14s          
   |++++++++++++++++++++++++++++++++++++++++++++++    | 91% ~12s          
   |++++++++++++++++++++++++++++++++++++++++++++++   | 92% ~11s          
   |+++++++++++++++++++++++++++++++++++++++++++++++   | 93% ~10s          
   |+++++++++++++++++++++++++++++++++++++++++++++++   | 94% ~08s          
   |++++++++++++++++++++++++++++++++++++++++++++++++  | 95% ~07s          
   |++++++++++++++++++++++++++++++++++++++++++++++++  | 96% ~06s          
   |+++++++++++++++++++++++++++++++++++++++++++++++++ | 97% ~04s          
   |+++++++++++++++++++++++++++++++++++++++++++++++++ | 98% ~03s          
   |++++++++++++++++++++++++++++++++++++++++++++++++++| 99% ~01s          
   |++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed = 02m 18s

   |                                                  | 0 % ~calculating  
   |+                                                 | 1 % ~52s          
   |++                                                | 2 % ~45s          
   |++                                                | 4 % ~48s          
   |+++                                               | 5 % ~47s          
   |++++                                              | 6 % ~45s          
   |++++                                              | 7 % ~44s          
   |+++++                                             | 9 % ~43s          
   |+++++                                             | 10% ~43s          
   |++++++                                            | 11% ~42s          
   |+++++++                                           | 12% ~41s          
   |+++++++                                           | 14% ~40s          
   |++++++++                                          | 15% ~39s          
   |+++++++++                                         | 16% ~38s          
   |+++++++++                                         | 17% ~37s          
   |++++++++++                                        | 19% ~36s          
   |++++++++++                                        | 20% ~36s          
   |+++++++++++                                       | 21% ~35s          
   |++++++++++++                                      | 22% ~36s          
   |++++++++++++                                      | 23% ~35s          
   |+++++++++++++                                     | 25% ~35s          
   |+++++++++++++                                     | 26% ~37s          
   |++++++++++++++                                    | 27% ~36s          
   |+++++++++++++++                                   | 28% ~35s          
   |+++++++++++++++                                   | 30% ~34s          
   |++++++++++++++++                                  | 31% ~33s          
   |+++++++++++++++++                                 | 32% ~33s          
   |+++++++++++++++++                                 | 33% ~32s          
   |++++++++++++++++++                                | 35% ~31s          
   |++++++++++++++++++                                | 36% ~31s          
   |+++++++++++++++++++                               | 37% ~30s          
   |++++++++++++++++++++                              | 38% ~29s          
   |++++++++++++++++++++                              | 40% ~28s          
   |+++++++++++++++++++++                             | 41% ~28s          
   |+++++++++++++++++++++                             | 42% ~27s          
   |++++++++++++++++++++++                            | 43% ~27s          
   |+++++++++++++++++++++++                           | 44% ~26s          
   |+++++++++++++++++++++++                           | 46% ~25s          
   |++++++++++++++++++++++++                          | 47% ~25s          
   |+++++++++++++++++++++++++                         | 48% ~24s          
   |+++++++++++++++++++++++++                         | 49% ~24s          
   |++++++++++++++++++++++++++                        | 51% ~23s          
   |++++++++++++++++++++++++++                        | 52% ~22s          
   |+++++++++++++++++++++++++++                       | 53% ~22s          
   |++++++++++++++++++++++++++++                      | 54% ~21s          
   |++++++++++++++++++++++++++++                      | 56% ~20s          
   |+++++++++++++++++++++++++++++                     | 57% ~20s          
   |++++++++++++++++++++++++++++++                    | 58% ~19s          
   |++++++++++++++++++++++++++++++                    | 59% ~19s          
   |+++++++++++++++++++++++++++++++                   | 60% ~18s          
   |+++++++++++++++++++++++++++++++                   | 62% ~18s          
   |++++++++++++++++++++++++++++++++                  | 63% ~17s          
   |+++++++++++++++++++++++++++++++++                 | 64% ~16s          
   |+++++++++++++++++++++++++++++++++                 | 65% ~16s          
   |++++++++++++++++++++++++++++++++++                | 67% ~15s          
   |++++++++++++++++++++++++++++++++++                | 68% ~15s          
   |+++++++++++++++++++++++++++++++++++               | 69% ~14s          
   |++++++++++++++++++++++++++++++++++++              | 70% ~13s          
   |++++++++++++++++++++++++++++++++++++              | 72% ~13s          
   |+++++++++++++++++++++++++++++++++++++             | 73% ~12s          
   |++++++++++++++++++++++++++++++++++++++            | 74% ~12s          
   |++++++++++++++++++++++++++++++++++++++            | 75% ~12s          
   |+++++++++++++++++++++++++++++++++++++++           | 77% ~11s          
   |+++++++++++++++++++++++++++++++++++++++           | 78% ~10s          
   |++++++++++++++++++++++++++++++++++++++++          | 79% ~10s          
   |+++++++++++++++++++++++++++++++++++++++++         | 80% ~09s          
   |+++++++++++++++++++++++++++++++++++++++++         | 81% ~09s          
   |++++++++++++++++++++++++++++++++++++++++++        | 83% ~08s          
   |++++++++++++++++++++++++++++++++++++++++++        | 84% ~07s          
   |+++++++++++++++++++++++++++++++++++++++++++       | 85% ~07s          
   |++++++++++++++++++++++++++++++++++++++++++++      | 86% ~06s          
   |++++++++++++++++++++++++++++++++++++++++++++      | 88% ~06s          
   |+++++++++++++++++++++++++++++++++++++++++++++     | 89% ~05s          
   |++++++++++++++++++++++++++++++++++++++++++++++    | 90% ~05s          
   |++++++++++++++++++++++++++++++++++++++++++++++    | 91% ~04s          
   |+++++++++++++++++++++++++++++++++++++++++++++++   | 93% ~03s          
   |+++++++++++++++++++++++++++++++++++++++++++++++   | 94% ~03s          
   |++++++++++++++++++++++++++++++++++++++++++++++++  | 95% ~02s          
   |+++++++++++++++++++++++++++++++++++++++++++++++++ | 96% ~02s          
   |+++++++++++++++++++++++++++++++++++++++++++++++++ | 98% ~01s          
   |++++++++++++++++++++++++++++++++++++++++++++++++++| 99% ~01s          
   |++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed = 46s

   |                                                  | 0 % ~calculating  
   |+                                                 | 1 % ~01m 33s      
   |++                                                | 3 % ~01m 15s      
   |+++                                               | 4 % ~01m 06s      
   |+++                                               | 6 % ~01m 02s      
   |++++                                              | 7 % ~59s          
   |+++++                                             | 8 % ~56s          
   |+++++                                             | 10% ~55s          
   |++++++                                            | 11% ~53s          
   |+++++++                                           | 12% ~52s          
   |+++++++                                           | 14% ~50s          
   |++++++++                                          | 15% ~49s          
   |+++++++++                                         | 17% ~48s          
   |++++++++++                                        | 18% ~47s          
   |++++++++++                                        | 19% ~47s          
   |+++++++++++                                       | 21% ~46s          
   |++++++++++++                                      | 22% ~46s          
   |++++++++++++                                      | 24% ~45s          
   |+++++++++++++                                     | 25% ~47s          
   |++++++++++++++                                    | 26% ~49s          
   |++++++++++++++                                    | 28% ~48s          
   |+++++++++++++++                                   | 29% ~47s          
   |++++++++++++++++                                  | 31% ~46s          
   |++++++++++++++++                                  | 32% ~46s          
   |+++++++++++++++++                                 | 33% ~47s          
   |++++++++++++++++++                                | 35% ~45s          
   |+++++++++++++++++++                               | 36% ~43s          
   |+++++++++++++++++++                               | 38% ~42s          
   |++++++++++++++++++++                              | 39% ~41s          
   |+++++++++++++++++++++                             | 40% ~39s          
   |+++++++++++++++++++++                             | 42% ~38s          
   |++++++++++++++++++++++                            | 43% ~37s          
   |+++++++++++++++++++++++                           | 44% ~36s          
   |+++++++++++++++++++++++                           | 46% ~35s          
   |++++++++++++++++++++++++                          | 47% ~34s          
   |+++++++++++++++++++++++++                         | 49% ~33s          
   |+++++++++++++++++++++++++                         | 50% ~32s          
   |++++++++++++++++++++++++++                        | 51% ~31s          
   |+++++++++++++++++++++++++++                       | 53% ~30s          
   |++++++++++++++++++++++++++++                      | 54% ~29s          
   |++++++++++++++++++++++++++++                      | 56% ~29s          
   |+++++++++++++++++++++++++++++                     | 57% ~28s          
   |++++++++++++++++++++++++++++++                    | 58% ~27s          
   |++++++++++++++++++++++++++++++                    | 60% ~26s          
   |+++++++++++++++++++++++++++++++                   | 61% ~25s          
   |++++++++++++++++++++++++++++++++                  | 62% ~24s          
   |++++++++++++++++++++++++++++++++                  | 64% ~23s          
   |+++++++++++++++++++++++++++++++++                 | 65% ~22s          
   |++++++++++++++++++++++++++++++++++                | 67% ~21s          
   |+++++++++++++++++++++++++++++++++++               | 68% ~20s          
   |+++++++++++++++++++++++++++++++++++               | 69% ~19s          
   |++++++++++++++++++++++++++++++++++++              | 71% ~18s          
   |+++++++++++++++++++++++++++++++++++++             | 72% ~17s          
   |+++++++++++++++++++++++++++++++++++++             | 74% ~17s          
   |++++++++++++++++++++++++++++++++++++++            | 75% ~16s          
   |+++++++++++++++++++++++++++++++++++++++           | 76% ~15s          
   |+++++++++++++++++++++++++++++++++++++++           | 78% ~14s          
   |++++++++++++++++++++++++++++++++++++++++          | 79% ~13s          
   |+++++++++++++++++++++++++++++++++++++++++         | 81% ~12s          
   |+++++++++++++++++++++++++++++++++++++++++         | 82% ~11s          
   |++++++++++++++++++++++++++++++++++++++++++        | 83% ~11s          
   |+++++++++++++++++++++++++++++++++++++++++++       | 85% ~10s          
   |++++++++++++++++++++++++++++++++++++++++++++      | 86% ~09s          
   |++++++++++++++++++++++++++++++++++++++++++++      | 88% ~08s          
   |+++++++++++++++++++++++++++++++++++++++++++++     | 89% ~07s          
   |++++++++++++++++++++++++++++++++++++++++++++++    | 90% ~06s          
   |++++++++++++++++++++++++++++++++++++++++++++++    | 92% ~05s          
   |+++++++++++++++++++++++++++++++++++++++++++++++   | 93% ~04s          
   |++++++++++++++++++++++++++++++++++++++++++++++++  | 94% ~04s          
   |++++++++++++++++++++++++++++++++++++++++++++++++  | 96% ~03s          
   |+++++++++++++++++++++++++++++++++++++++++++++++++ | 97% ~02s          
   |++++++++++++++++++++++++++++++++++++++++++++++++++| 99% ~01s          
   |++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed = 01m 05s

   |                                                  | 0 % ~calculating  
   |+                                                 | 1 % ~01m 52s      
   |++                                                | 2 % ~01m 41s      
   |++                                                | 4 % ~01m 51s      
   |+++                                               | 5 % ~01m 44s      
   |+++                                               | 6 % ~01m 37s      
   |++++                                              | 7 % ~01m 45s      
   |+++++                                             | 8 % ~01m 36s      
   |+++++                                             | 10% ~01m 29s      
   |++++++                                            | 11% ~01m 24s      
   |++++++                                            | 12% ~01m 19s      
   |+++++++                                           | 13% ~01m 15s      
   |++++++++                                          | 14% ~01m 13s      
   |++++++++                                          | 15% ~01m 11s      
   |+++++++++                                         | 17% ~01m 10s      
   |+++++++++                                         | 18% ~01m 10s      
   |++++++++++                                        | 19% ~01m 08s      
   |+++++++++++                                       | 20% ~01m 07s      
   |+++++++++++                                       | 21% ~01m 06s      
   |++++++++++++                                      | 23% ~01m 05s      
   |++++++++++++                                      | 24% ~01m 03s      
   |+++++++++++++                                     | 25% ~01m 02s      
   |++++++++++++++                                    | 26% ~01m 02s      
   |++++++++++++++                                    | 27% ~01m 00s      
   |+++++++++++++++                                   | 29% ~59s          
   |+++++++++++++++                                   | 30% ~58s          
   |++++++++++++++++                                  | 31% ~57s          
   |+++++++++++++++++                                 | 32% ~56s          
   |+++++++++++++++++                                 | 33% ~55s          
   |++++++++++++++++++                                | 35% ~53s          
   |++++++++++++++++++                                | 36% ~54s          
   |+++++++++++++++++++                               | 37% ~52s          
   |++++++++++++++++++++                              | 38% ~51s          
   |++++++++++++++++++++                              | 39% ~50s          
   |+++++++++++++++++++++                             | 40% ~49s          
   |+++++++++++++++++++++                             | 42% ~47s          
   |++++++++++++++++++++++                            | 43% ~46s          
   |+++++++++++++++++++++++                           | 44% ~45s          
   |+++++++++++++++++++++++                           | 45% ~44s          
   |++++++++++++++++++++++++                          | 46% ~43s          
   |++++++++++++++++++++++++                          | 48% ~41s          
   |+++++++++++++++++++++++++                         | 49% ~40s          
   |+++++++++++++++++++++++++                         | 50% ~39s          
   |++++++++++++++++++++++++++                        | 51% ~38s          
   |+++++++++++++++++++++++++++                       | 52% ~37s          
   |+++++++++++++++++++++++++++                       | 54% ~37s          
   |++++++++++++++++++++++++++++                      | 55% ~36s          
   |++++++++++++++++++++++++++++                      | 56% ~39s          
   |+++++++++++++++++++++++++++++                     | 57% ~38s          
   |++++++++++++++++++++++++++++++                    | 58% ~36s          
   |++++++++++++++++++++++++++++++                    | 60% ~35s          
   |+++++++++++++++++++++++++++++++                   | 61% ~34s          
   |+++++++++++++++++++++++++++++++                   | 62% ~33s          
   |++++++++++++++++++++++++++++++++                  | 63% ~32s          
   |+++++++++++++++++++++++++++++++++                 | 64% ~31s          
   |+++++++++++++++++++++++++++++++++                 | 65% ~29s          
   |++++++++++++++++++++++++++++++++++                | 67% ~28s          
   |++++++++++++++++++++++++++++++++++                | 68% ~27s          
   |+++++++++++++++++++++++++++++++++++               | 69% ~26s          
   |++++++++++++++++++++++++++++++++++++              | 70% ~25s          
   |++++++++++++++++++++++++++++++++++++              | 71% ~24s          
   |+++++++++++++++++++++++++++++++++++++             | 73% ~23s          
   |+++++++++++++++++++++++++++++++++++++             | 74% ~22s          
   |++++++++++++++++++++++++++++++++++++++            | 75% ~21s          
   |+++++++++++++++++++++++++++++++++++++++           | 76% ~20s          
   |+++++++++++++++++++++++++++++++++++++++           | 77% ~19s          
   |++++++++++++++++++++++++++++++++++++++++          | 79% ~18s          
   |++++++++++++++++++++++++++++++++++++++++          | 80% ~17s          
   |+++++++++++++++++++++++++++++++++++++++++         | 81% ~16s          
   |++++++++++++++++++++++++++++++++++++++++++        | 82% ~15s          
   |++++++++++++++++++++++++++++++++++++++++++        | 83% ~14s          
   |+++++++++++++++++++++++++++++++++++++++++++       | 85% ~13s          
   |+++++++++++++++++++++++++++++++++++++++++++       | 86% ~12s          
   |++++++++++++++++++++++++++++++++++++++++++++      | 87% ~11s          
   |+++++++++++++++++++++++++++++++++++++++++++++     | 88% ~10s          
   |+++++++++++++++++++++++++++++++++++++++++++++     | 89% ~09s          
   |++++++++++++++++++++++++++++++++++++++++++++++    | 90% ~08s          
   |++++++++++++++++++++++++++++++++++++++++++++++    | 92% ~07s          
   |+++++++++++++++++++++++++++++++++++++++++++++++   | 93% ~06s          
   |++++++++++++++++++++++++++++++++++++++++++++++++  | 94% ~05s          
   |++++++++++++++++++++++++++++++++++++++++++++++++  | 95% ~04s          
   |+++++++++++++++++++++++++++++++++++++++++++++++++ | 96% ~03s          
   |+++++++++++++++++++++++++++++++++++++++++++++++++ | 98% ~02s          
   |++++++++++++++++++++++++++++++++++++++++++++++++++| 99% ~01s          
   |++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed = 01m 20s

   |                                                  | 0 % ~calculating  
   |+                                                 | 1 % ~04m 29s      
   |++                                                | 2 % ~03m 31s      
   |++                                                | 3 % ~03m 03s      
   |+++                                               | 4 % ~02m 51s      
   |+++                                               | 5 % ~02m 43s      
   |++++                                              | 6 % ~02m 34s      
   |++++                                              | 7 % ~02m 28s      
   |+++++                                             | 8 % ~02m 26s      
   |+++++                                             | 9 % ~03m 00s      
   |++++++                                            | 10% ~02m 51s      
   |++++++                                            | 11% ~02m 42s      
   |+++++++                                           | 12% ~02m 36s      
   |+++++++                                           | 13% ~02m 30s      
   |++++++++                                          | 14% ~02m 26s      
   |++++++++                                          | 15% ~02m 22s      
   |+++++++++                                         | 16% ~02m 21s      
   |+++++++++                                         | 17% ~02m 17s      
   |++++++++++                                        | 18% ~02m 14s      
   |++++++++++                                        | 19% ~02m 11s      
   |+++++++++++                                       | 20% ~02m 08s      
   |+++++++++++                                       | 21% ~02m 05s      
   |++++++++++++                                      | 22% ~02m 03s      
   |++++++++++++                                      | 23% ~02m 01s      
   |+++++++++++++                                     | 24% ~01m 59s      
   |+++++++++++++                                     | 25% ~01m 56s      
   |++++++++++++++                                    | 26% ~01m 54s      
   |++++++++++++++                                    | 27% ~01m 52s      
   |+++++++++++++++                                   | 28% ~01m 50s      
   |+++++++++++++++                                   | 29% ~01m 47s      
   |++++++++++++++++                                  | 30% ~01m 45s      
   |++++++++++++++++                                  | 31% ~01m 43s      
   |+++++++++++++++++                                 | 32% ~01m 42s      
   |+++++++++++++++++                                 | 33% ~01m 40s      
   |++++++++++++++++++                                | 34% ~01m 39s      
   |++++++++++++++++++                                | 35% ~01m 37s      
   |+++++++++++++++++++                               | 36% ~01m 36s      
   |+++++++++++++++++++                               | 37% ~01m 35s      
   |++++++++++++++++++++                              | 38% ~01m 33s      
   |++++++++++++++++++++                              | 39% ~01m 31s      
   |+++++++++++++++++++++                             | 40% ~01m 30s      
   |+++++++++++++++++++++                             | 41% ~01m 29s      
   |++++++++++++++++++++++                            | 42% ~01m 27s      
   |++++++++++++++++++++++                            | 43% ~01m 25s      
   |+++++++++++++++++++++++                           | 44% ~01m 23s      
   |+++++++++++++++++++++++                           | 45% ~01m 21s      
   |++++++++++++++++++++++++                          | 46% ~01m 20s      
   |++++++++++++++++++++++++                          | 47% ~01m 18s      
   |+++++++++++++++++++++++++                         | 48% ~01m 16s      
   |+++++++++++++++++++++++++                         | 49% ~01m 14s      
   |++++++++++++++++++++++++++                        | 51% ~01m 13s      
   |++++++++++++++++++++++++++                        | 52% ~01m 11s      
   |+++++++++++++++++++++++++++                       | 53% ~01m 09s      
   |+++++++++++++++++++++++++++                       | 54% ~01m 08s      
   |++++++++++++++++++++++++++++                      | 55% ~01m 06s      
   |++++++++++++++++++++++++++++                      | 56% ~01m 04s      
   |+++++++++++++++++++++++++++++                     | 57% ~01m 03s      
   |+++++++++++++++++++++++++++++                     | 58% ~01m 01s      
   |++++++++++++++++++++++++++++++                    | 59% ~60s          
   |++++++++++++++++++++++++++++++                    | 60% ~58s          
   |+++++++++++++++++++++++++++++++                   | 61% ~57s          
   |+++++++++++++++++++++++++++++++                   | 62% ~56s          
   |++++++++++++++++++++++++++++++++                  | 63% ~54s          
   |++++++++++++++++++++++++++++++++                  | 64% ~53s          
   |+++++++++++++++++++++++++++++++++                 | 65% ~51s          
   |+++++++++++++++++++++++++++++++++                 | 66% ~50s          
   |++++++++++++++++++++++++++++++++++                | 67% ~48s          
   |++++++++++++++++++++++++++++++++++                | 68% ~47s          
   |+++++++++++++++++++++++++++++++++++               | 69% ~45s          
   |+++++++++++++++++++++++++++++++++++               | 70% ~44s          
   |++++++++++++++++++++++++++++++++++++              | 71% ~42s          
   |++++++++++++++++++++++++++++++++++++              | 72% ~41s          
   |+++++++++++++++++++++++++++++++++++++             | 73% ~39s          
   |+++++++++++++++++++++++++++++++++++++             | 74% ~38s          
   |++++++++++++++++++++++++++++++++++++++            | 75% ~37s          
   |++++++++++++++++++++++++++++++++++++++            | 76% ~35s          
   |+++++++++++++++++++++++++++++++++++++++           | 77% ~34s          
   |+++++++++++++++++++++++++++++++++++++++           | 78% ~32s          
   |++++++++++++++++++++++++++++++++++++++++          | 79% ~31s          
   |++++++++++++++++++++++++++++++++++++++++          | 80% ~29s          
   |+++++++++++++++++++++++++++++++++++++++++         | 81% ~28s          
   |+++++++++++++++++++++++++++++++++++++++++         | 82% ~26s          
   |++++++++++++++++++++++++++++++++++++++++++        | 83% ~25s          
   |++++++++++++++++++++++++++++++++++++++++++        | 84% ~23s          
   |+++++++++++++++++++++++++++++++++++++++++++       | 85% ~22s          
   |+++++++++++++++++++++++++++++++++++++++++++       | 86% ~20s          
   |++++++++++++++++++++++++++++++++++++++++++++      | 87% ~19s          
   |++++++++++++++++++++++++++++++++++++++++++++      | 88% ~18s          
   |+++++++++++++++++++++++++++++++++++++++++++++     | 89% ~16s          
   |+++++++++++++++++++++++++++++++++++++++++++++     | 90% ~15s          
   |++++++++++++++++++++++++++++++++++++++++++++++    | 91% ~13s          
   |++++++++++++++++++++++++++++++++++++++++++++++    | 92% ~12s          
   |+++++++++++++++++++++++++++++++++++++++++++++++   | 93% ~10s          
   |+++++++++++++++++++++++++++++++++++++++++++++++   | 94% ~09s          
   |++++++++++++++++++++++++++++++++++++++++++++++++  | 95% ~07s          
   |++++++++++++++++++++++++++++++++++++++++++++++++  | 96% ~06s          
   |+++++++++++++++++++++++++++++++++++++++++++++++++ | 97% ~04s          
   |+++++++++++++++++++++++++++++++++++++++++++++++++ | 98% ~03s          
   |++++++++++++++++++++++++++++++++++++++++++++++++++| 99% ~01s          
   |++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed = 02m 25s

   |                                                  | 0 % ~calculating  
   |+                                                 | 1 % ~02m 05s      
   |++                                                | 2 % ~01m 59s      
   |++                                                | 3 % ~01m 57s      
   |+++                                               | 4 % ~01m 56s      
   |+++                                               | 5 % ~01m 57s      
   |++++                                              | 6 % ~01m 55s      
   |++++                                              | 7 % ~01m 56s      
   |+++++                                             | 9 % ~01m 53s      
   |+++++                                             | 10% ~01m 56s      
   |++++++                                            | 11% ~01m 56s      
   |++++++                                            | 12% ~01m 56s      
   |+++++++                                           | 13% ~01m 55s      
   |+++++++                                           | 14% ~01m 58s      
   |++++++++                                          | 15% ~01m 57s      
   |++++++++                                          | 16% ~01m 56s      
   |+++++++++                                         | 17% ~01m 54s      
   |++++++++++                                        | 18% ~01m 52s      
   |++++++++++                                        | 19% ~01m 52s      
   |+++++++++++                                       | 20% ~01m 51s      
   |+++++++++++                                       | 21% ~01m 49s      
   |++++++++++++                                      | 22% ~01m 47s      
   |++++++++++++                                      | 23% ~01m 45s      
   |+++++++++++++                                     | 24% ~01m 43s      
   |+++++++++++++                                     | 26% ~01m 41s      
   |++++++++++++++                                    | 27% ~01m 39s      
   |++++++++++++++                                    | 28% ~01m 40s      
   |+++++++++++++++                                   | 29% ~01m 38s      
   |+++++++++++++++                                   | 30% ~01m 36s      
   |++++++++++++++++                                  | 31% ~01m 34s      
   |++++++++++++++++                                  | 32% ~01m 35s      
   |+++++++++++++++++                                 | 33% ~01m 34s      
   |++++++++++++++++++                                | 34% ~01m 32s      
   |++++++++++++++++++                                | 35% ~01m 30s      
   |+++++++++++++++++++                               | 36% ~01m 29s      
   |+++++++++++++++++++                               | 37% ~01m 27s      
   |++++++++++++++++++++                              | 38% ~01m 25s      
   |++++++++++++++++++++                              | 39% ~01m 23s      
   |+++++++++++++++++++++                             | 40% ~01m 22s      
   |+++++++++++++++++++++                             | 41% ~01m 21s      
   |++++++++++++++++++++++                            | 43% ~01m 19s      
   |++++++++++++++++++++++                            | 44% ~01m 18s      
   |+++++++++++++++++++++++                           | 45% ~01m 17s      
   |+++++++++++++++++++++++                           | 46% ~01m 17s      
   |++++++++++++++++++++++++                          | 47% ~01m 15s      
   |++++++++++++++++++++++++                          | 48% ~01m 14s      
   |+++++++++++++++++++++++++                         | 49% ~01m 12s      
   |+++++++++++++++++++++++++                         | 50% ~01m 11s      
   |++++++++++++++++++++++++++                        | 51% ~01m 09s      
   |+++++++++++++++++++++++++++                       | 52% ~01m 08s      
   |+++++++++++++++++++++++++++                       | 53% ~01m 06s      
   |++++++++++++++++++++++++++++                      | 54% ~01m 04s      
   |++++++++++++++++++++++++++++                      | 55% ~01m 03s      
   |+++++++++++++++++++++++++++++                     | 56% ~01m 03s      
   |+++++++++++++++++++++++++++++                     | 57% ~01m 01s      
   |++++++++++++++++++++++++++++++                    | 59% ~59s          
   |++++++++++++++++++++++++++++++                    | 60% ~58s          
   |+++++++++++++++++++++++++++++++                   | 61% ~56s          
   |+++++++++++++++++++++++++++++++                   | 62% ~56s          
   |++++++++++++++++++++++++++++++++                  | 63% ~54s          
   |++++++++++++++++++++++++++++++++                  | 64% ~53s          
   |+++++++++++++++++++++++++++++++++                 | 65% ~51s          
   |+++++++++++++++++++++++++++++++++                 | 66% ~50s          
   |++++++++++++++++++++++++++++++++++                | 67% ~48s          
   |+++++++++++++++++++++++++++++++++++               | 68% ~47s          
   |+++++++++++++++++++++++++++++++++++               | 69% ~45s          
   |++++++++++++++++++++++++++++++++++++              | 70% ~46s          
   |++++++++++++++++++++++++++++++++++++              | 71% ~44s          
   |+++++++++++++++++++++++++++++++++++++             | 72% ~42s          
   |+++++++++++++++++++++++++++++++++++++             | 73% ~40s          
   |++++++++++++++++++++++++++++++++++++++            | 74% ~39s          
   |++++++++++++++++++++++++++++++++++++++            | 76% ~37s          
   |+++++++++++++++++++++++++++++++++++++++           | 77% ~35s          
   |+++++++++++++++++++++++++++++++++++++++           | 78% ~34s          
   |++++++++++++++++++++++++++++++++++++++++          | 79% ~32s          
   |++++++++++++++++++++++++++++++++++++++++          | 80% ~30s          
   |+++++++++++++++++++++++++++++++++++++++++         | 81% ~29s          
   |+++++++++++++++++++++++++++++++++++++++++         | 82% ~27s          
   |++++++++++++++++++++++++++++++++++++++++++        | 83% ~25s          
   |+++++++++++++++++++++++++++++++++++++++++++       | 84% ~24s          
   |+++++++++++++++++++++++++++++++++++++++++++       | 85% ~22s          
   |++++++++++++++++++++++++++++++++++++++++++++      | 86% ~21s          
   |++++++++++++++++++++++++++++++++++++++++++++      | 87% ~19s          
   |+++++++++++++++++++++++++++++++++++++++++++++     | 88% ~17s          
   |+++++++++++++++++++++++++++++++++++++++++++++     | 89% ~16s          
   |++++++++++++++++++++++++++++++++++++++++++++++    | 90% ~14s          
   |++++++++++++++++++++++++++++++++++++++++++++++    | 91% ~13s          
   |+++++++++++++++++++++++++++++++++++++++++++++++   | 93% ~11s          
   |+++++++++++++++++++++++++++++++++++++++++++++++   | 94% ~09s          
   |++++++++++++++++++++++++++++++++++++++++++++++++  | 95% ~08s          
   |++++++++++++++++++++++++++++++++++++++++++++++++  | 96% ~06s          
   |+++++++++++++++++++++++++++++++++++++++++++++++++ | 97% ~05s          
   |+++++++++++++++++++++++++++++++++++++++++++++++++ | 98% ~03s          
   |++++++++++++++++++++++++++++++++++++++++++++++++++| 99% ~02s          
   |++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed = 02m 25s

   |                                                  | 0 % ~calculating  
   |+                                                 | 1 % ~43s          
   |++                                                | 3 % ~34s          
   |++                                                | 4 % ~31s          
   |+++                                               | 5 % ~29s          
   |++++                                              | 6 % ~28s          
   |++++                                              | 8 % ~31s          
   |+++++                                             | 9 % ~32s          
   |++++++                                            | 10% ~30s          
   |++++++                                            | 12% ~29s          
   |+++++++                                           | 13% ~28s          
   |++++++++                                          | 14% ~26s          
   |++++++++                                          | 16% ~25s          
   |+++++++++                                         | 17% ~24s          
   |++++++++++                                        | 18% ~23s          
   |++++++++++                                        | 19% ~23s          
   |+++++++++++                                       | 21% ~22s          
   |++++++++++++                                      | 22% ~22s          
   |++++++++++++                                      | 23% ~21s          
   |+++++++++++++                                     | 25% ~20s          
   |+++++++++++++                                     | 26% ~20s          
   |++++++++++++++                                    | 27% ~19s          
   |+++++++++++++++                                   | 29% ~19s          
   |+++++++++++++++                                   | 30% ~18s          
   |++++++++++++++++                                  | 31% ~18s          
   |+++++++++++++++++                                 | 32% ~18s          
   |+++++++++++++++++                                 | 34% ~17s          
   |++++++++++++++++++                                | 35% ~19s          
   |+++++++++++++++++++                               | 36% ~18s          
   |+++++++++++++++++++                               | 38% ~18s          
   |++++++++++++++++++++                              | 39% ~17s          
   |+++++++++++++++++++++                             | 40% ~17s          
   |+++++++++++++++++++++                             | 42% ~17s          
   |++++++++++++++++++++++                            | 43% ~16s          
   |+++++++++++++++++++++++                           | 44% ~16s          
   |+++++++++++++++++++++++                           | 45% ~16s          
   |++++++++++++++++++++++++                          | 47% ~15s          
   |+++++++++++++++++++++++++                         | 48% ~15s          
   |+++++++++++++++++++++++++                         | 49% ~14s          
   |++++++++++++++++++++++++++                        | 51% ~14s          
   |++++++++++++++++++++++++++                        | 52% ~13s          
   |+++++++++++++++++++++++++++                       | 53% ~13s          
   |++++++++++++++++++++++++++++                      | 55% ~12s          
   |++++++++++++++++++++++++++++                      | 56% ~12s          
   |+++++++++++++++++++++++++++++                     | 57% ~11s          
   |++++++++++++++++++++++++++++++                    | 58% ~11s          
   |++++++++++++++++++++++++++++++                    | 60% ~11s          
   |+++++++++++++++++++++++++++++++                   | 61% ~10s          
   |++++++++++++++++++++++++++++++++                  | 62% ~10s          
   |++++++++++++++++++++++++++++++++                  | 64% ~09s          
   |+++++++++++++++++++++++++++++++++                 | 65% ~09s          
   |++++++++++++++++++++++++++++++++++                | 66% ~09s          
   |++++++++++++++++++++++++++++++++++                | 68% ~08s          
   |+++++++++++++++++++++++++++++++++++               | 69% ~08s          
   |++++++++++++++++++++++++++++++++++++              | 70% ~08s          
   |++++++++++++++++++++++++++++++++++++              | 71% ~07s          
   |+++++++++++++++++++++++++++++++++++++             | 73% ~07s          
   |++++++++++++++++++++++++++++++++++++++            | 74% ~07s          
   |++++++++++++++++++++++++++++++++++++++            | 75% ~06s          
   |+++++++++++++++++++++++++++++++++++++++           | 77% ~06s          
   |+++++++++++++++++++++++++++++++++++++++           | 78% ~06s          
   |++++++++++++++++++++++++++++++++++++++++          | 79% ~05s          
   |+++++++++++++++++++++++++++++++++++++++++         | 81% ~05s          
   |+++++++++++++++++++++++++++++++++++++++++         | 82% ~05s          
   |++++++++++++++++++++++++++++++++++++++++++        | 83% ~04s          
   |+++++++++++++++++++++++++++++++++++++++++++       | 84% ~04s          
   |+++++++++++++++++++++++++++++++++++++++++++       | 86% ~04s          
   |++++++++++++++++++++++++++++++++++++++++++++      | 87% ~03s          
   |+++++++++++++++++++++++++++++++++++++++++++++     | 88% ~03s          
   |+++++++++++++++++++++++++++++++++++++++++++++     | 90% ~03s          
   |++++++++++++++++++++++++++++++++++++++++++++++    | 91% ~02s          
   |+++++++++++++++++++++++++++++++++++++++++++++++   | 92% ~02s          
   |+++++++++++++++++++++++++++++++++++++++++++++++   | 94% ~02s          
   |++++++++++++++++++++++++++++++++++++++++++++++++  | 95% ~01s          
   |+++++++++++++++++++++++++++++++++++++++++++++++++ | 96% ~01s          
   |+++++++++++++++++++++++++++++++++++++++++++++++++ | 97% ~01s          
   |++++++++++++++++++++++++++++++++++++++++++++++++++| 99% ~00s          
   |++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed = 25s

   |                                                  | 0 % ~calculating  
   |+                                                 | 1 % ~02m 04s      
   |++                                                | 2 % ~02m 00s      
   |++                                                | 3 % ~01m 57s      
   |+++                                               | 4 % ~01m 59s      
   |+++                                               | 5 % ~02m 19s      
   |++++                                              | 6 % ~02m 17s      
   |++++                                              | 7 % ~02m 15s      
   |+++++                                             | 8 % ~02m 14s      
   |+++++                                             | 9 % ~02m 11s      
   |++++++                                            | 10% ~02m 09s      
   |++++++                                            | 11% ~02m 07s      
   |+++++++                                           | 12% ~02m 04s      
   |+++++++                                           | 13% ~02m 02s      
   |++++++++                                          | 14% ~02m 02s      
   |++++++++                                          | 15% ~02m 02s      
   |+++++++++                                         | 16% ~01m 60s      
   |+++++++++                                         | 18% ~02m 03s      
   |++++++++++                                        | 19% ~02m 00s      
   |++++++++++                                        | 20% ~01m 58s      
   |+++++++++++                                       | 21% ~01m 56s      
   |+++++++++++                                       | 22% ~01m 53s      
   |++++++++++++                                      | 23% ~01m 52s      
   |++++++++++++                                      | 24% ~01m 50s      
   |+++++++++++++                                     | 25% ~01m 48s      
   |+++++++++++++                                     | 26% ~01m 47s      
   |++++++++++++++                                    | 27% ~01m 45s      
   |++++++++++++++                                    | 28% ~01m 44s      
   |+++++++++++++++                                   | 29% ~01m 45s      
   |+++++++++++++++                                   | 30% ~01m 43s      
   |++++++++++++++++                                  | 31% ~01m 41s      
   |++++++++++++++++                                  | 32% ~01m 40s      
   |+++++++++++++++++                                 | 33% ~01m 38s      
   |++++++++++++++++++                                | 34% ~01m 37s      
   |++++++++++++++++++                                | 35% ~01m 35s      
   |+++++++++++++++++++                               | 36% ~01m 43s      
   |+++++++++++++++++++                               | 37% ~01m 41s      
   |++++++++++++++++++++                              | 38% ~01m 38s      
   |++++++++++++++++++++                              | 39% ~01m 37s      
   |+++++++++++++++++++++                             | 40% ~01m 35s      
   |+++++++++++++++++++++                             | 41% ~01m 32s      
   |++++++++++++++++++++++                            | 42% ~01m 30s      
   |++++++++++++++++++++++                            | 43% ~01m 28s      
   |+++++++++++++++++++++++                           | 44% ~01m 26s      
   |+++++++++++++++++++++++                           | 45% ~01m 24s      
   |++++++++++++++++++++++++                          | 46% ~01m 22s      
   |++++++++++++++++++++++++                          | 47% ~01m 20s      
   |+++++++++++++++++++++++++                         | 48% ~01m 18s      
   |+++++++++++++++++++++++++                         | 49% ~01m 17s      
   |++++++++++++++++++++++++++                        | 51% ~01m 15s      
   |++++++++++++++++++++++++++                        | 52% ~01m 13s      
   |+++++++++++++++++++++++++++                       | 53% ~01m 12s      
   |+++++++++++++++++++++++++++                       | 54% ~01m 10s      
   |++++++++++++++++++++++++++++                      | 55% ~01m 08s      
   |++++++++++++++++++++++++++++                      | 56% ~01m 07s      
   |+++++++++++++++++++++++++++++                     | 57% ~01m 05s      
   |+++++++++++++++++++++++++++++                     | 58% ~01m 04s      
   |++++++++++++++++++++++++++++++                    | 59% ~01m 02s      
   |++++++++++++++++++++++++++++++                    | 60% ~01m 00s      
   |+++++++++++++++++++++++++++++++                   | 61% ~59s          
   |+++++++++++++++++++++++++++++++                   | 62% ~57s          
   |++++++++++++++++++++++++++++++++                  | 63% ~56s          
   |++++++++++++++++++++++++++++++++                  | 64% ~54s          
   |+++++++++++++++++++++++++++++++++                 | 65% ~52s          
   |+++++++++++++++++++++++++++++++++                 | 66% ~51s          
   |++++++++++++++++++++++++++++++++++                | 67% ~49s          
   |+++++++++++++++++++++++++++++++++++               | 68% ~48s          
   |+++++++++++++++++++++++++++++++++++               | 69% ~47s          
   |++++++++++++++++++++++++++++++++++++              | 70% ~45s          
   |++++++++++++++++++++++++++++++++++++              | 71% ~43s          
   |+++++++++++++++++++++++++++++++++++++             | 72% ~42s          
   |+++++++++++++++++++++++++++++++++++++             | 73% ~40s          
   |++++++++++++++++++++++++++++++++++++++            | 74% ~38s          
   |++++++++++++++++++++++++++++++++++++++            | 75% ~37s          
   |+++++++++++++++++++++++++++++++++++++++           | 76% ~35s          
   |+++++++++++++++++++++++++++++++++++++++           | 77% ~34s          
   |++++++++++++++++++++++++++++++++++++++++          | 78% ~32s          
   |++++++++++++++++++++++++++++++++++++++++          | 79% ~31s          
   |+++++++++++++++++++++++++++++++++++++++++         | 80% ~29s          
   |+++++++++++++++++++++++++++++++++++++++++         | 81% ~27s          
   |++++++++++++++++++++++++++++++++++++++++++        | 82% ~26s          
   |++++++++++++++++++++++++++++++++++++++++++        | 84% ~25s          
   |+++++++++++++++++++++++++++++++++++++++++++       | 85% ~23s          
   |+++++++++++++++++++++++++++++++++++++++++++       | 86% ~21s          
   |++++++++++++++++++++++++++++++++++++++++++++      | 87% ~20s          
   |++++++++++++++++++++++++++++++++++++++++++++      | 88% ~18s          
   |+++++++++++++++++++++++++++++++++++++++++++++     | 89% ~17s          
   |+++++++++++++++++++++++++++++++++++++++++++++     | 90% ~15s          
   |++++++++++++++++++++++++++++++++++++++++++++++    | 91% ~14s          
   |++++++++++++++++++++++++++++++++++++++++++++++    | 92% ~12s          
   |+++++++++++++++++++++++++++++++++++++++++++++++   | 93% ~11s          
   |+++++++++++++++++++++++++++++++++++++++++++++++   | 94% ~09s          
   |++++++++++++++++++++++++++++++++++++++++++++++++  | 95% ~08s          
   |++++++++++++++++++++++++++++++++++++++++++++++++  | 96% ~06s          
   |+++++++++++++++++++++++++++++++++++++++++++++++++ | 97% ~05s          
   |+++++++++++++++++++++++++++++++++++++++++++++++++ | 98% ~03s          
   |++++++++++++++++++++++++++++++++++++++++++++++++++| 99% ~02s          
   |++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed = 02m 28s

   |                                                  | 0 % ~calculating  
   |+                                                 | 1 % ~03m 02s      
   |++                                                | 2 % ~02m 57s      
   |++                                                | 3 % ~02m 50s      
   |+++                                               | 4 % ~02m 43s      
   |+++                                               | 5 % ~02m 44s      
   |++++                                              | 7 % ~02m 44s      
   |++++                                              | 8 % ~02m 42s      
   |+++++                                             | 9 % ~02m 40s      
   |+++++                                             | 10% ~02m 38s      
   |++++++                                            | 11% ~02m 35s      
   |++++++                                            | 12% ~02m 36s      
   |+++++++                                           | 13% ~02m 35s      
   |++++++++                                          | 14% ~02m 41s      
   |++++++++                                          | 15% ~02m 37s      
   |+++++++++                                         | 16% ~02m 33s      
   |+++++++++                                         | 17% ~02m 29s      
   |++++++++++                                        | 18% ~02m 25s      
   |++++++++++                                        | 20% ~02m 22s      
   |+++++++++++                                       | 21% ~02m 21s      
   |+++++++++++                                       | 22% ~02m 19s      
   |++++++++++++                                      | 23% ~02m 17s      
   |++++++++++++                                      | 24% ~02m 14s      
   |+++++++++++++                                     | 25% ~02m 14s      
   |++++++++++++++                                    | 26% ~02m 11s      
   |++++++++++++++                                    | 27% ~02m 09s      
   |+++++++++++++++                                   | 28% ~02m 07s      
   |+++++++++++++++                                   | 29% ~02m 05s      
   |++++++++++++++++                                  | 30% ~02m 04s      
   |++++++++++++++++                                  | 32% ~02m 01s      
   |+++++++++++++++++                                 | 33% ~01m 59s      
   |+++++++++++++++++                                 | 34% ~01m 57s      
   |++++++++++++++++++                                | 35% ~01m 57s      
   |++++++++++++++++++                                | 36% ~01m 55s      
   |+++++++++++++++++++                               | 37% ~01m 53s      
   |++++++++++++++++++++                              | 38% ~01m 51s      
   |++++++++++++++++++++                              | 39% ~01m 49s      
   |+++++++++++++++++++++                             | 40% ~01m 47s      
   |+++++++++++++++++++++                             | 41% ~01m 45s      
   |++++++++++++++++++++++                            | 42% ~01m 43s      
   |++++++++++++++++++++++                            | 43% ~01m 40s      
   |+++++++++++++++++++++++                           | 45% ~01m 38s      
   |+++++++++++++++++++++++                           | 46% ~01m 36s      
   |++++++++++++++++++++++++                          | 47% ~01m 34s      
   |++++++++++++++++++++++++                          | 48% ~01m 32s      
   |+++++++++++++++++++++++++                         | 49% ~01m 32s      
   |+++++++++++++++++++++++++                         | 50% ~01m 29s      
   |++++++++++++++++++++++++++                        | 51% ~01m 27s      
   |+++++++++++++++++++++++++++                       | 52% ~01m 25s      
   |+++++++++++++++++++++++++++                       | 53% ~01m 23s      
   |++++++++++++++++++++++++++++                      | 54% ~01m 21s      
   |++++++++++++++++++++++++++++                      | 55% ~01m 19s      
   |+++++++++++++++++++++++++++++                     | 57% ~01m 17s      
   |+++++++++++++++++++++++++++++                     | 58% ~01m 15s      
   |++++++++++++++++++++++++++++++                    | 59% ~01m 13s      
   |++++++++++++++++++++++++++++++                    | 60% ~01m 11s      
   |+++++++++++++++++++++++++++++++                   | 61% ~01m 12s      
   |+++++++++++++++++++++++++++++++                   | 62% ~01m 10s      
   |++++++++++++++++++++++++++++++++                  | 63% ~01m 08s      
   |+++++++++++++++++++++++++++++++++                 | 64% ~01m 05s      
   |+++++++++++++++++++++++++++++++++                 | 65% ~01m 03s      
   |++++++++++++++++++++++++++++++++++                | 66% ~01m 01s      
   |++++++++++++++++++++++++++++++++++                | 67% ~59s          
   |+++++++++++++++++++++++++++++++++++               | 68% ~57s          
   |+++++++++++++++++++++++++++++++++++               | 70% ~55s          
   |++++++++++++++++++++++++++++++++++++              | 71% ~53s          
   |++++++++++++++++++++++++++++++++++++              | 72% ~51s          
   |+++++++++++++++++++++++++++++++++++++             | 73% ~49s          
   |+++++++++++++++++++++++++++++++++++++             | 74% ~47s          
   |++++++++++++++++++++++++++++++++++++++            | 75% ~45s          
   |+++++++++++++++++++++++++++++++++++++++           | 76% ~43s          
   |+++++++++++++++++++++++++++++++++++++++           | 77% ~41s          
   |++++++++++++++++++++++++++++++++++++++++          | 78% ~39s          
   |++++++++++++++++++++++++++++++++++++++++          | 79% ~37s          
   |+++++++++++++++++++++++++++++++++++++++++         | 80% ~35s          
   |+++++++++++++++++++++++++++++++++++++++++         | 82% ~33s          
   |++++++++++++++++++++++++++++++++++++++++++        | 83% ~31s          
   |++++++++++++++++++++++++++++++++++++++++++        | 84% ~29s          
   |+++++++++++++++++++++++++++++++++++++++++++       | 85% ~27s          
   |+++++++++++++++++++++++++++++++++++++++++++       | 86% ~25s          
   |++++++++++++++++++++++++++++++++++++++++++++      | 87% ~23s          
   |+++++++++++++++++++++++++++++++++++++++++++++     | 88% ~21s          
   |+++++++++++++++++++++++++++++++++++++++++++++     | 89% ~19s          
   |++++++++++++++++++++++++++++++++++++++++++++++    | 90% ~17s          
   |++++++++++++++++++++++++++++++++++++++++++++++    | 91% ~15s          
   |+++++++++++++++++++++++++++++++++++++++++++++++   | 92% ~14s          
   |+++++++++++++++++++++++++++++++++++++++++++++++   | 93% ~12s          
   |++++++++++++++++++++++++++++++++++++++++++++++++  | 95% ~10s          
   |++++++++++++++++++++++++++++++++++++++++++++++++  | 96% ~08s          
   |+++++++++++++++++++++++++++++++++++++++++++++++++ | 97% ~06s          
   |+++++++++++++++++++++++++++++++++++++++++++++++++ | 98% ~04s          
   |++++++++++++++++++++++++++++++++++++++++++++++++++| 99% ~02s          
   |++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed = 02m 56s

   |                                                  | 0 % ~calculating  
   |+                                                 | 1 % ~01m 46s      
   |++                                                | 2 % ~01m 45s      
   |++                                                | 3 % ~01m 53s      
   |+++                                               | 4 % ~01m 51s      
   |+++                                               | 5 % ~01m 49s      
   |++++                                              | 7 % ~01m 46s      
   |++++                                              | 8 % ~01m 48s      
   |+++++                                             | 9 % ~01m 46s      
   |+++++                                             | 10% ~01m 47s      
   |++++++                                            | 11% ~01m 47s      
   |+++++++                                           | 12% ~01m 45s      
   |+++++++                                           | 13% ~01m 45s      
   |++++++++                                          | 14% ~01m 43s      
   |++++++++                                          | 15% ~01m 41s      
   |+++++++++                                         | 16% ~01m 40s      
   |+++++++++                                         | 18% ~01m 40s      
   |++++++++++                                        | 19% ~01m 40s      
   |++++++++++                                        | 20% ~01m 39s      
   |+++++++++++                                       | 21% ~01m 38s      
   |+++++++++++                                       | 22% ~01m 40s      
   |++++++++++++                                      | 23% ~01m 39s      
   |+++++++++++++                                     | 24% ~01m 37s      
   |+++++++++++++                                     | 25% ~01m 36s      
   |++++++++++++++                                    | 26% ~01m 34s      
   |++++++++++++++                                    | 27% ~01m 32s      
   |+++++++++++++++                                   | 29% ~01m 30s      
   |+++++++++++++++                                   | 30% ~01m 29s      
   |++++++++++++++++                                  | 31% ~01m 28s      
   |++++++++++++++++                                  | 32% ~01m 27s      
   |+++++++++++++++++                                 | 33% ~01m 26s      
   |++++++++++++++++++                                | 34% ~01m 24s      
   |++++++++++++++++++                                | 35% ~01m 23s      
   |+++++++++++++++++++                               | 36% ~01m 23s      
   |+++++++++++++++++++                               | 37% ~01m 22s      
   |++++++++++++++++++++                              | 38% ~01m 20s      
   |++++++++++++++++++++                              | 40% ~01m 19s      
   |+++++++++++++++++++++                             | 41% ~01m 17s      
   |+++++++++++++++++++++                             | 42% ~01m 16s      
   |++++++++++++++++++++++                            | 43% ~01m 14s      
   |++++++++++++++++++++++                            | 44% ~01m 13s      
   |+++++++++++++++++++++++                           | 45% ~01m 12s      
   |++++++++++++++++++++++++                          | 46% ~01m 11s      
   |++++++++++++++++++++++++                          | 47% ~01m 11s      
   |+++++++++++++++++++++++++                         | 48% ~01m 09s      
   |+++++++++++++++++++++++++                         | 49% ~01m 08s      
   |++++++++++++++++++++++++++                        | 51% ~01m 06s      
   |++++++++++++++++++++++++++                        | 52% ~01m 05s      
   |+++++++++++++++++++++++++++                       | 53% ~01m 03s      
   |+++++++++++++++++++++++++++                       | 54% ~01m 01s      
   |++++++++++++++++++++++++++++                      | 55% ~60s          
   |+++++++++++++++++++++++++++++                     | 56% ~58s          
   |+++++++++++++++++++++++++++++                     | 57% ~57s          
   |++++++++++++++++++++++++++++++                    | 58% ~56s          
   |++++++++++++++++++++++++++++++                    | 59% ~54s          
   |+++++++++++++++++++++++++++++++                   | 60% ~53s          
   |+++++++++++++++++++++++++++++++                   | 62% ~51s          
   |++++++++++++++++++++++++++++++++                  | 63% ~50s          
   |++++++++++++++++++++++++++++++++                  | 64% ~49s          
   |+++++++++++++++++++++++++++++++++                 | 65% ~48s          
   |+++++++++++++++++++++++++++++++++                 | 66% ~46s          
   |++++++++++++++++++++++++++++++++++                | 67% ~45s          
   |+++++++++++++++++++++++++++++++++++               | 68% ~43s          
   |+++++++++++++++++++++++++++++++++++               | 69% ~42s          
   |++++++++++++++++++++++++++++++++++++              | 70% ~40s          
   |++++++++++++++++++++++++++++++++++++              | 71% ~39s          
   |+++++++++++++++++++++++++++++++++++++             | 73% ~37s          
   |+++++++++++++++++++++++++++++++++++++             | 74% ~36s          
   |++++++++++++++++++++++++++++++++++++++            | 75% ~35s          
   |++++++++++++++++++++++++++++++++++++++            | 76% ~33s          
   |+++++++++++++++++++++++++++++++++++++++           | 77% ~32s          
   |++++++++++++++++++++++++++++++++++++++++          | 78% ~30s          
   |++++++++++++++++++++++++++++++++++++++++          | 79% ~29s          
   |+++++++++++++++++++++++++++++++++++++++++         | 80% ~27s          
   |+++++++++++++++++++++++++++++++++++++++++         | 81% ~26s          
   |++++++++++++++++++++++++++++++++++++++++++        | 82% ~24s          
   |++++++++++++++++++++++++++++++++++++++++++        | 84% ~23s          
   |+++++++++++++++++++++++++++++++++++++++++++       | 85% ~22s          
   |+++++++++++++++++++++++++++++++++++++++++++       | 86% ~20s          
   |++++++++++++++++++++++++++++++++++++++++++++      | 87% ~19s          
   |++++++++++++++++++++++++++++++++++++++++++++      | 88% ~17s          
   |+++++++++++++++++++++++++++++++++++++++++++++     | 89% ~15s          
   |++++++++++++++++++++++++++++++++++++++++++++++    | 90% ~14s          
   |++++++++++++++++++++++++++++++++++++++++++++++    | 91% ~12s          
   |+++++++++++++++++++++++++++++++++++++++++++++++   | 92% ~11s          
   |+++++++++++++++++++++++++++++++++++++++++++++++   | 93% ~09s          
   |++++++++++++++++++++++++++++++++++++++++++++++++  | 95% ~08s          
   |++++++++++++++++++++++++++++++++++++++++++++++++  | 96% ~06s          
   |+++++++++++++++++++++++++++++++++++++++++++++++++ | 97% ~05s          
   |+++++++++++++++++++++++++++++++++++++++++++++++++ | 98% ~03s          
   |++++++++++++++++++++++++++++++++++++++++++++++++++| 99% ~02s          
   |++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed = 02m 18s
# What about using a different test?  On your own time, trying adding the argument test.use = "roc" and see how different your results are. 

Visualize differentially expressed genes

# The list of DE genes is long and it would be too much information to visualize them all. So, let's plot the top 5 features (genes) for each cluster.  
# call library dplyr to use some of its functions
library(dplyr)
package ‘dplyr’ was built under R version 3.5.1
Attaching package: ‘dplyr’

The following objects are masked from ‘package:stats’:

    filter, lag

The following objects are masked from ‘package:base’:

    intersect, setdiff, setequal, union
# group by cluster using group_by function and show top 5 results using top_n function
pbmc.markers.grouped <- group_by(pbmc.markers, cluster)
pbmc.markers.grouped.top <- top_n(pbmc.markers.grouped, 5, avg_logFC)
# convert it to a matrix for downstream applications (trust me)
pbmc.markers.grouped.top <- as.matrix(pbmc.markers.grouped.top)
print(pbmc.markers.grouped.top)
      p_val           avg_logFC  pct.1   pct.2   p_val_adj       cluster gene      
 [1,] " 0.000000e+00" "1.360231" "0.688" "0.181" " 0.000000e+00" "0"     "LTB"     
 [2,] " 0.000000e+00" "1.247725" "0.678" "0.224" " 0.000000e+00" "0"     "LDHB"    
 [3,] " 0.000000e+00" "1.236988" "0.446" "0.104" " 0.000000e+00" "0"     "SELL"    
 [4,] " 0.000000e+00" "1.151767" "0.962" "0.783" " 0.000000e+00" "0"     "PABPC1"  
 [5,] " 0.000000e+00" "1.059882" "0.445" "0.125" " 0.000000e+00" "0"     "IL7R"    
 [6,] " 0.000000e+00" "2.965944" "0.852" "0.050" " 0.000000e+00" "1"     "S100A8"  
 [7,] " 0.000000e+00" "2.926204" "0.468" "0.038" " 0.000000e+00" "1"     "CCL2"    
 [8,] " 0.000000e+00" "2.769285" "0.880" "0.091" " 0.000000e+00" "1"     "IL8"     
 [9,] " 0.000000e+00" "2.730468" "0.762" "0.043" " 0.000000e+00" "1"     "S100A9"  
[10,] " 0.000000e+00" "2.405877" "1.000" "0.872" " 0.000000e+00" "1"     "FTL"     
[11,] " 0.000000e+00" "2.144159" "0.478" "0.032" " 0.000000e+00" "2"     "GZMH"    
[12,] " 0.000000e+00" "2.072391" "0.861" "0.126" " 0.000000e+00" "2"     "CCL5"    
[13,] " 0.000000e+00" "1.651099" "0.402" "0.038" " 0.000000e+00" "2"     "CD8A"    
[14,] " 0.000000e+00" "1.572040" "0.743" "0.107" " 0.000000e+00" "2"     "NKG7"    
[15,] " 0.000000e+00" "1.562925" "0.554" "0.082" " 0.000000e+00" "2"     "GZMB"    
[16,] " 0.000000e+00" "2.869341" "0.814" "0.020" " 0.000000e+00" "3"     "CD79A"   
[17,] " 0.000000e+00" "2.139385" "0.519" "0.009" " 0.000000e+00" "3"     "MS4A1"   
[18,] " 0.000000e+00" "2.132632" "0.998" "0.503" " 0.000000e+00" "3"     "CD74"    
[19,] " 0.000000e+00" "1.783456" "0.527" "0.077" " 0.000000e+00" "3"     "CD83"    
[20,] " 0.000000e+00" "1.776907" "0.401" "0.012" " 0.000000e+00" "3"     "CD79B"   
[21,] " 0.000000e+00" "3.521768" "0.945" "0.070" " 0.000000e+00" "4"     "GNLY"    
[22,] " 0.000000e+00" "2.607033" "0.950" "0.111" " 0.000000e+00" "4"     "NKG7"    
[23,] " 0.000000e+00" "2.490249" "0.845" "0.075" " 0.000000e+00" "4"     "GZMB"    
[24,] " 0.000000e+00" "2.446398" "0.569" "0.025" " 0.000000e+00" "4"     "CLIC3"   
[25,] " 0.000000e+00" "2.242966" "0.520" "0.042" " 0.000000e+00" "4"     "FGFBP2"  
[26,] " 0.000000e+00" "2.900754" "0.620" "0.036" " 0.000000e+00" "5"     "VMO1"    
[27,] " 0.000000e+00" "2.344538" "0.269" "0.006" " 0.000000e+00" "5"     "CXCL10"  
[28,] " 0.000000e+00" "2.296334" "0.841" "0.129" " 0.000000e+00" "5"     "FCGR3A"  
[29,] " 0.000000e+00" "2.146760" "0.824" "0.111" " 0.000000e+00" "5"     "MS4A7"   
[30,] " 0.000000e+00" "1.830722" "0.698" "0.067" " 0.000000e+00" "5"     "IFITM3"  
[31,] "2.858001e-294" "2.183459" "0.487" "0.067" "3.868876e-290" "6"     "HSPH1"   
[32,] "2.541939e-202" "2.142275" "0.677" "0.218" "3.441023e-198" "6"     "CACYBP"  
[33,] "8.179524e-148" "1.961616" "0.362" "0.070" "1.107262e-143" "6"     "DNAJB1"  
[34,] "2.725095e-146" "2.582634" "0.478" "0.134" "3.688962e-142" "6"     "HSPB1"   
[35,] " 2.001307e-90" "2.390347" "0.344" "0.098" " 2.709169e-86" "6"     "HSPA1A"  
[36,] " 0.000000e+00" "3.547369" "0.714" "0.032" " 0.000000e+00" "7"     "PPBP"    
[37,] " 0.000000e+00" "2.581009" "0.522" "0.007" " 0.000000e+00" "7"     "PF4"     
[38,] " 0.000000e+00" "2.513364" "0.528" "0.012" " 0.000000e+00" "7"     "GNG11"   
[39,] " 0.000000e+00" "2.213129" "0.457" "0.007" " 0.000000e+00" "7"     "SDPR"    
[40,] " 0.000000e+00" "1.827362" "0.298" "0.002" " 0.000000e+00" "7"     "TUBB1"   
[41,] " 0.000000e+00" "2.375324" "0.934" "0.118" " 0.000000e+00" "8"     "HLA-DQA1"
[42,] "4.022389e-211" "2.344441" "0.985" "0.303" "5.445108e-207" "8"     "HLA-DPA1"
[43,] "3.958257e-187" "2.249284" "0.996" "0.413" "5.358293e-183" "8"     "HLA-DRA" 
[44,] "4.185786e-176" "2.420208" "0.865" "0.219" "5.666298e-172" "8"     "LYZ"     
[45,] "2.606548e-143" "2.736058" "0.894" "0.262" "3.528484e-139" "8"     "TXN"     
[46,] " 0.000000e+00" "3.082401" "0.797" "0.015" " 0.000000e+00" "9"     "MIR155HG"
[47,] " 0.000000e+00" "2.140877" "0.758" "0.025" " 0.000000e+00" "9"     "NME1"    
[48,] "9.557452e-207" "2.246809" "0.922" "0.126" "1.293792e-202" "9"     "HLA-DQA1"
[49,] "2.175653e-183" "1.924433" "0.843" "0.110" "2.945181e-179" "9"     "CD83"    
[50,] "2.271334e-143" "2.287105" "0.673" "0.089" "3.074705e-139" "9"     "MYC"     
[51,] " 0.000000e+00" "6.778140" "1.000" "0.017" " 0.000000e+00" "10"    "HBA2"    
[52,] " 0.000000e+00" "6.221867" "1.000" "0.010" " 0.000000e+00" "10"    "HBA1"    
[53,] " 0.000000e+00" "2.675141" "0.780" "0.000" " 0.000000e+00" "10"    "ALAS2"   
[54,] "2.298284e-175" "2.029726" "0.780" "0.028" "3.111187e-171" "10"    "SNCA"    
[55,] "5.222162e-117" "7.546049" "1.000" "0.081" "7.069241e-113" "10"    "HBB"     
# First, isolate the top 5 DE genes for each cluster.
cluster_0 <- which(pbmc.markers.grouped.top[,"cluster"] == 0)
cluster_1 <- which(pbmc.markers.grouped.top[,"cluster"] == 1)
cluster_2 <- which(pbmc.markers.grouped.top[,"cluster"] == 2)
cluster_3 <- which(pbmc.markers.grouped.top[,"cluster"] == 3)
cluster_4 <- which(pbmc.markers.grouped.top[,"cluster"] == 4)
cluster_5 <- which(pbmc.markers.grouped.top[,"cluster"] == 5)
cluster_6 <- which(pbmc.markers.grouped.top[,"cluster"] == 6)
cluster_7 <- which(pbmc.markers.grouped.top[,"cluster"] == 7)
cluster_8 <- which(pbmc.markers.grouped.top[,"cluster"] == 8)
cluster_9 <- which(pbmc.markers.grouped.top[,"cluster"] == 9)
cluster_10 <- which(pbmc.markers.grouped.top[,"cluster"] == 10)
# 3 options for visualization...
# 1. Violin plots
# if you want to see the points, insert point.size.use = 0.01 into the argument list
# # specify cluster 0 here
VlnPlot(object = pbmc, features.plot = pbmc.markers.grouped.top[cluster_0, "gene"],
        point.size.use = 0)

# # specify cluster 1 here
VlnPlot(object = pbmc, features.plot = pbmc.markers.grouped.top[cluster_1, "gene"],
        point.size.use = 0)

# # specify cluster 2 here
VlnPlot(object = pbmc, features.plot = pbmc.markers.grouped.top[cluster_2, "gene"],
        point.size.use = 0)

# # specify cluster 3 here
VlnPlot(object = pbmc, features.plot = pbmc.markers.grouped.top[cluster_3, "gene"],
        point.size.use = 0)

# # specify cluster 4 here
VlnPlot(object = pbmc, features.plot = pbmc.markers.grouped.top[cluster_4, "gene"],
        point.size.use = 0)

# # specify cluster 5 here
VlnPlot(object = pbmc, features.plot = pbmc.markers.grouped.top[cluster_5, "gene"],
        point.size.use = 0)

# # specify cluster 6 here
VlnPlot(object = pbmc, features.plot = pbmc.markers.grouped.top[cluster_6, "gene"],
        point.size.use = 0)

# # specify cluster 7 here
VlnPlot(object = pbmc, features.plot = pbmc.markers.grouped.top[cluster_7, "gene"],
        point.size.use = 0)

# # specify cluster 8 here
VlnPlot(object = pbmc, features.plot = pbmc.markers.grouped.top[cluster_8, "gene"],
        point.size.use = 0)

# # specify cluster 9 here
VlnPlot(object = pbmc, features.plot = pbmc.markers.grouped.top[cluster_9, "gene"],
        point.size.use = 0)

# # specify cluster 10 here
VlnPlot(object = pbmc, features.plot = pbmc.markers.grouped.top[cluster_10, "gene"],
        point.size.use = 0)

# 2. Feature plot
# You could change the reduction.use to do PCA or ICA instead of t-SNE, but since we know where our clusters were mapped in the context of t-SNE, this makes the most sense for us.
# Cluster 0
FeaturePlot(object = pbmc, features.plot = pbmc.markers.grouped.top[cluster_0, "gene"], cols.use = c("grey", "blue"), reduction.use = "tsne")

# Cluster 1
FeaturePlot(object = pbmc, features.plot = pbmc.markers.grouped.top[cluster_1, "gene"], cols.use = c("grey", "blue"), reduction.use = "tsne")

# Cluster 2
FeaturePlot(object = pbmc, features.plot = pbmc.markers.grouped.top[cluster_2, "gene"], cols.use = c("grey", "blue"), reduction.use = "tsne")

# Cluster 3
FeaturePlot(object = pbmc, features.plot = pbmc.markers.grouped.top[cluster_3, "gene"], cols.use = c("grey", "blue"), reduction.use = "tsne")

# Cluster 4
FeaturePlot(object = pbmc, features.plot = pbmc.markers.grouped.top[cluster_4, "gene"], cols.use = c("grey", "blue"), reduction.use = "tsne")

# Cluster 5
FeaturePlot(object = pbmc, features.plot = pbmc.markers.grouped.top[cluster_5, "gene"], cols.use = c("grey", "blue"), reduction.use = "tsne")

# Cluster 6
FeaturePlot(object = pbmc, features.plot = pbmc.markers.grouped.top[cluster_6, "gene"], cols.use = c("grey", "blue"), reduction.use = "tsne")

# Cluster 7
FeaturePlot(object = pbmc, features.plot = pbmc.markers.grouped.top[cluster_7, "gene"], cols.use = c("grey", "blue"), reduction.use = "tsne")

# Cluster 8
FeaturePlot(object = pbmc, features.plot = pbmc.markers.grouped.top[cluster_8, "gene"], cols.use = c("grey", "blue"), reduction.use = "tsne")

# Cluster 9
FeaturePlot(object = pbmc, features.plot = pbmc.markers.grouped.top[cluster_9, "gene"], cols.use = c("grey", "blue"), reduction.use = "tsne")

# Cluster 10
FeaturePlot(object = pbmc, features.plot = pbmc.markers.grouped.top[cluster_10, "gene"], cols.use = c("grey", "blue"), reduction.use = "tsne")

# 3. Heatmap
# Heatmaps have enough room to show you the top 5 (or more!) genes for each cluster.  Let's do the top 5, but if you have time, try changing the top features to 10 or more.
DoHeatmap(object = pbmc, genes.use = pbmc.markers.grouped.top[ , "gene"], slim.col.label = TRUE, remove.key = TRUE)

Save the object and workspace

# save this object and come back to it later
saveRDS(pbmc, file = paste(directory, "/pbmc_only.rds", sep = ""))

save.image(paste(directory, “/section1.RData”, sep = ""))

Now let’s add in a second dataset, which is PBMCs stimulated with interferon gamma

# load in the data - how many cells and how many genes?  use dim(stim.data) to find out
stim.data <- read.table(paste(directory, "/immune_stimulated_expression_matrix.txt.gz", sep = ""), sep = "\t")
# create the Seurat object for the stimulated PBMCs
stim <- CreateSeuratObject(raw.data = stim.data, min.cells = 5, project = "stim_PBMC")
# Also re-create the control PBMC object because we will change a couple things this time and we don't want to have the previous info stored in the Seurat object.
pbmc <- CreateSeuratObject(raw.data = pbmc.data, min.cells = 5, project = "control_PBMC")

We have a few goals in this section of the analysis.

1. identify cell types present in both the control pbmc and stimulated pbmc datasets
2. find markers that identify cell types that are conserved in control and stimulated pbmcs
3. compare the datasets to find cell-type specific responses to interferon stimulation

Really, the goal is to reinforce the major steps in scRNA-seq analysis by going over some of them again. This time, we show how even a simple experimental design (control vs stimulated) can complicate the analysis.

Filtering, normalization, scaling, and finding HVGs

VlnPlot(object = stim, features.plot = c("nGene", "nUMI"), nCol = 2, point.size.use = 0.001)

GenePlot(object = stim, gene1 = "nUMI", gene2 = "nGene")

# Filter out the stimulated pbmcs with low total gene expression
  # for this analysis, we want only the genes with at least average nGene
stim <- FilterCells(object = stim, subset.names = "nGene", low.thresholds = 500, high.thresholds = 1800)
# normalize
stim <- NormalizeData(stim)
Performing log-normalization
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
# scale
stim <- ScaleData(stim)
Scaling data matrix

  |                                                                                      
  |                                                                                |   0%
  |                                                                                      
  |======                                                                          |   7%
  |                                                                                      
  |===========                                                                     |  14%
  |                                                                                      
  |=================                                                               |  21%
  |                                                                                      
  |=======================                                                         |  29%
  |                                                                                      
  |=============================                                                   |  36%
  |                                                                                      
  |==================================                                              |  43%
  |                                                                                      
  |========================================                                        |  50%
  |                                                                                      
  |==============================================                                  |  57%
  |                                                                                      
  |===================================================                             |  64%
  |                                                                                      
  |=========================================================                       |  71%
  |                                                                                      
  |===============================================================                 |  79%
  |                                                                                      
  |=====================================================================           |  86%
  |                                                                                      
  |==========================================================================      |  93%
  |                                                                                      
  |================================================================================| 100%
# Find variable genes
stim <-FindVariableGenes(stim)
Calculating gene means
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
Calculating gene variance to mean ratios
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|

# make metadata indicating it is stimulated
stim@meta.data$stim <- "STIM"
# yes, we already did this for the control dataset, but the downstream analysis requires a bit more selective filtering for it to run.  The only changes we are making is choosing cells with 500-1800 genes instead of 200-1800 genes.
pbmc <- FilterCells(object = pbmc, subset.names = "nGene", low.thresholds = 500, high.thresholds = 1800)
# normalize
pbmc <- NormalizeData(pbmc)
Performing log-normalization
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
# scale
pbmc <- ScaleData(pbmc)
Scaling data matrix

  |                                                                                      
  |                                                                                |   0%
  |                                                                                      
  |======                                                                          |   7%
  |                                                                                      
  |===========                                                                     |  14%
  |                                                                                      
  |=================                                                               |  21%
  |                                                                                      
  |=======================                                                         |  29%
  |                                                                                      
  |=============================                                                   |  36%
  |                                                                                      
  |==================================                                              |  43%
  |                                                                                      
  |========================================                                        |  50%
  |                                                                                      
  |==============================================                                  |  57%
  |                                                                                      
  |===================================================                             |  64%
  |                                                                                      
  |=========================================================                       |  71%
  |                                                                                      
  |===============================================================                 |  79%
  |                                                                                      
  |=====================================================================           |  86%
  |                                                                                      
  |==========================================================================      |  93%
  |                                                                                      
  |================================================================================| 100%
# Find variable genes
pbmc <-FindVariableGenes(pbmc)
Calculating gene means
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
Calculating gene variance to mean ratios
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|

# make metadata indicating it is stimulated
pbmc@meta.data$stim <- "CTRL"
# Now find the highly variable genes that are shared between the two datasets (control and stimulated)
g.1 <- head(rownames(pbmc@hvg.info), 1000)
g.2 <- head(rownames(stim@hvg.info), 1000)
# keeps only unique genenames (gets rid of duplicates)
genes.use <- unique(c(g.1, g.2))
genes.use <- intersect(genes.use, rownames(pbmc@scale.data))
genes.use <- intersect(genes.use, rownames(stim@scale.data))

Canonical Correlation Analysis (CCA)

# Run CCA to identify common sources of variation between the two datasets
# CCA finds linear combinations of features across the conditions that are maximally           correlated.  In short, it searches for patterns in the datasets that are common.
immune.combined <- RunCCA(pbmc, stim, genes.use = genes.use, num.cc = 30)
Running CCA
Merging objects
Performing log-normalization
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
Scaling data matrix

  |                                                                                      
  |                                                                                |   0%
  |                                                                                      
  |=====                                                                           |   7%
  |                                                                                      
  |===========                                                                     |  13%
  |                                                                                      
  |================                                                                |  20%
  |                                                                                      
  |=====================                                                           |  27%
  |                                                                                      
  |===========================                                                     |  33%
  |                                                                                      
  |================================                                                |  40%
  |                                                                                      
  |=====================================                                           |  47%
  |                                                                                      
  |===========================================                                     |  53%
  |                                                                                      
  |================================================                                |  60%
  |                                                                                      
  |=====================================================                           |  67%
  |                                                                                      
  |===========================================================                     |  73%
  |                                                                                      
  |================================================================                |  80%
  |                                                                                      
  |=====================================================================           |  87%
  |                                                                                      
  |===========================================================================     |  93%
  |                                                                                      
  |================================================================================| 100%
# Visualize CCA results
  # plot CC1 vs CC2
p1 <- DimPlot(object = immune.combined, reduction.use = "cca", group.by = "stim", pt.size = 0.5, do.return = TRUE)
  # violin plot to visualize CC1 and CC2
p2 <- VlnPlot(object = immune.combined, features.plot = "CC1", group.by = "stim", do.return = TRUE, point.size.use = 0.001)
p3 <- VlnPlot(object = immune.combined, features.plot = "CC2", group.by = "stim", do.return = TRUE, point.size.use = 0.001)
plot(p1)

plot_grid(p2, p3)

Choose CCs

# Much like selecting the number of PCs to use in the first analysis, we must select the number of CCs to use here.  The MetagenBicorPlot function below is analogous to the elbow plot method used in the first analysis of 1 dataset.  You look for saturation (flatline).
# 2 ways to pick CCs, but we won't run the first because it takes too long to compute for the time we have in class.  Feel free to try it on your own!
# # 1.  MetageneBicorPlot
p4 <- MetageneBicorPlot(immune.combined, grouping.var = "stim", dims.eval = 1:30, display.progress = TRUE)
Rescaling group 1
Scaling data matrix

  |                                                                                      
  |                                                                                |   0%
  |                                                                                      
  |===========================                                                     |  33%
  |                                                                                      
  |=====================================================                           |  67%
  |                                                                                      
  |================================================================================| 100%
Rescaling group 2
Scaling data matrix

  |                                                                                      
  |                                                                                |   0%
  |                                                                                      
  |===========================                                                     |  33%
  |                                                                                      
  |=====================================================                           |  67%
  |                                                                                      
  |================================================================================| 100%
Evaluating dims: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

  |                                                                                      
  |                                                                                |   0%
  |                                                                                      
  |===                                                                             |   3%
  |                                                                                      
  |=====                                                                           |   7%
  |                                                                                      
  |========                                                                        |  10%
  |                                                                                      
  |===========                                                                     |  13%
  |                                                                                      
  |=============                                                                   |  17%
  |                                                                                      
  |================                                                                |  20%
  |                                                                                      
  |===================                                                             |  23%
  |                                                                                      
  |=====================                                                           |  27%
  |                                                                                      
  |========================                                                        |  30%
  |                                                                                      
  |===========================                                                     |  33%
  |                                                                                      
  |=============================                                                   |  37%
  |                                                                                      
  |================================                                                |  40%
  |                                                                                      
  |===================================                                             |  43%
  |                                                                                      
  |=====================================                                           |  47%
  |                                                                                      
  |========================================                                        |  50%
  |                                                                                      
  |===========================================                                     |  53%
  |                                                                                      
  |=============================================                                   |  57%
  |                                                                                      
  |================================================                                |  60%
  |                                                                                      
  |===================================================                             |  63%
  |                                                                                      
  |=====================================================                           |  67%
  |                                                                                      
  |========================================================                        |  70%
  |                                                                                      
  |===========================================================                     |  73%
  |                                                                                      
  |=============================================================                   |  77%
  |                                                                                      
  |================================================================                |  80%
  |                                                                                      
  |===================================================================             |  83%
  |                                                                                      
  |=====================================================================           |  87%
  |                                                                                      
  |========================================================================        |  90%
  |                                                                                      
  |===========================================================================     |  93%
  |                                                                                      
  |=============================================================================   |  97%
  |                                                                                      
  |================================================================================| 100%

# 2. Heatmap - use the top 500 cells
  # you need three pages to see all the CCs
  # purple = low expression
  # yellow = high
DimHeatmap(object = immune.combined, reduction.type = "cca", cells.use = 500, dim.use = 1:9,
           do.balanced = TRUE)

DimHeatmap(object = immune.combined, reduction.type = "cca", cells.use = 500, dim.use = 10:18,
           do.balanced = TRUE)

DimHeatmap(object = immune.combined, reduction.type = "cca", cells.use = 500, dim.use = 19:30,
           do.balanced = TRUE)

# How many CCs do you think we should pick?

Align the CCA subspaces and visualize

# use the first 20 CCs to get a new dimension reduction that you can visualize and use for clustering
immune.combined <- AlignSubspace(immune.combined, reduction.type = "cca", grouping.var = "stim", 
                    dims.align = 1:20)
Rescaling group 1
Scaling data matrix

  |                                                                                      
  |                                                                                |   0%
  |                                                                                      
  |=====                                                                           |   7%
  |                                                                                      
  |===========                                                                     |  13%
  |                                                                                      
  |================                                                                |  20%
  |                                                                                      
  |=====================                                                           |  27%
  |                                                                                      
  |===========================                                                     |  33%
  |                                                                                      
  |================================                                                |  40%
  |                                                                                      
  |=====================================                                           |  47%
  |                                                                                      
  |===========================================                                     |  53%
  |                                                                                      
  |================================================                                |  60%
  |                                                                                      
  |=====================================================                           |  67%
  |                                                                                      
  |===========================================================                     |  73%
  |                                                                                      
  |================================================================                |  80%
  |                                                                                      
  |=====================================================================           |  87%
  |                                                                                      
  |===========================================================================     |  93%
  |                                                                                      
  |================================================================================| 100%
Rescaling group 2
Scaling data matrix

  |                                                                                      
  |                                                                                |   0%
  |                                                                                      
  |=====                                                                           |   7%
  |                                                                                      
  |===========                                                                     |  13%
  |                                                                                      
  |================                                                                |  20%
  |                                                                                      
  |=====================                                                           |  27%
  |                                                                                      
  |===========================                                                     |  33%
  |                                                                                      
  |================================                                                |  40%
  |                                                                                      
  |=====================================                                           |  47%
  |                                                                                      
  |===========================================                                     |  53%
  |                                                                                      
  |================================================                                |  60%
  |                                                                                      
  |=====================================================                           |  67%
  |                                                                                      
  |===========================================================                     |  73%
  |                                                                                      
  |================================================================                |  80%
  |                                                                                      
  |=====================================================================           |  87%
  |                                                                                      
  |===========================================================================     |  93%
  |                                                                                      
  |================================================================================| 100%
Aligning dimension 1

   |                                                  | 0 % ~calculating  
   |+                                                 | 1 % ~04h 35m 59s  

Find clusters using the aligned data

# Run t-SNE
immune.combined <- Seurat::RunTSNE(immune.combined, reduction.use = "cca.aligned", dims.use = 1:20, do.fast = T)

# cluster based on t-SNE space
immune.combined <- FindClusters(immune.combined, reduction.type = "cca.aligned", resolution = 0.6, dims.use = 1:20)

# Visualize the dataset alignment and combined clustering results on t-SNE
p1 <- TSNEPlot(immune.combined, do.return = T, pt.size = 0.5, group.by = "stim")
p2 <- TSNEPlot(immune.combined, do.label = T,  do.return = T, pt.size = 0.5)
plot_grid(p1, p2)

Differential gene expression of markers conserved across conditions


# Identify conserved cell type markers 
  # Conserved cell type makres are those that identify a particular cell type in both control       pbmc and stimulated pbmc).  The FindConservedmarkers function uses differential gene             expression analysis to do this.

# As an example, let's look for markers that are conserved between control and stimulated cells    in cluster 7, defined in the "ident.1" argument.
cluster7.markers <- FindConservedMarkers(immune.combined, ident.1 = 7, grouping.var = "stim", print.bar = FALSE)
# view the first few markers
cluster7.markers[1:9,]

# Confirm that those markers for cluster 7 identified fall into that location on t-SNE.
FeaturePlot(object = immune.combined, features.plot = rownames(cluster7.markers)[1:9], min.cutoff = "q9", cols.use = c("lightgrey", "blue"), pt.size = 0.5)

# Based on those results, what cell type is cluster 7?
# DO NOT RUN THIS MORE THAN ONCE

# Explore markers of different immune cell populations to try to identify what cell type each cluster contains.  Select candidates based off of prior knowledge of immune cell markers.
FeaturePlot(object = immune.combined, features.plot = c("CD3D", "SELL", "CREM", "CD8A", "GNLY", "CD79A", "FCGR3A", "CCL2", "PPBP"), min.cutoff = "q9", cols.use = c("lightgrey", "blue"), pt.size = 0.5)

# Now, you can rename your clusters as these cell names.  Trust that they are correct :)
new.ident <- c("CD14 Mono", "CD4 Naive T", "CD4 Memory T", "B", "CD16 Mono",
    "T activated", "CD8 T", "NK", "DC", "B activated", "Mk", "pDC", "Eryth")
for (i in 0:12) {
    immune.combined <- RenameIdent(object = immune.combined, old.ident.name = i,
        new.ident.name = new.ident[i + 1])
}

# Run t-SNE to see your new cluster names
TSNEPlot(immune.combined, do.label = T, pt.size = 0.5)

use SPlitDotPlotGG to plot markers of each cell type (using prior knowledge again).
immune.combined@ident <- factor(immune.combined@ident, levels = (c("pDC", "Eryth",
    "Mk", "DC", "CD14 Mono", "CD16 Mono", "B activated", "B", "CD8 T", "NK",
    "T activated", "CD4 Naive T", "CD4 Memory T")))
markers.to.plot <- c("CD3D", "CREM", "HSPH1", "SELL", "GIMAP5", "CACYBP", "GNLY",
    "NKG7", "CCL5", "CD8A", "MS4A1", "CD79A", "MIR155HG", "NME1", "FCGR3A",
    "VMO1", "CCL2", "S100A9", "HLA-DQA1", "GPR183", "PPBP", "GNG11", "HBA2",
    "HBB", "TSPAN13", "IL3RA", "IGJ")
sdp <- SplitDotPlotGG(immune.combined, genes.plot = rev(markers.to.plot), cols.use = c("blue", 
    "red"), x.lab.rot = T, plot.legend = T, dot.scale = 3, do.return = T, grouping.var = "stim")

Differential gene expression of markers that change across conditions (ctrl vs stimulated).

# It's great that we can identify conserved markers across datasets because it allows us to be confident that we have identified subpopulations of cell types that are present in both datasets, ctrl vs stimulated.  However, what is really interesting is what CHANGES when you stimulate the cells.  Let's find out.

# First we need to define some plotting functions to make labeling easier.  Don't worry about trying to understand what each line is doing, it is not interesting.  Just run it!

LabelPoint <- function(plot, genes, exp.mat, adj.x.t = 0, adj.y.t = 0, adj.x.s = 0, 
                       adj.y.s = 0, text.size = 2.5, segment.size = 0.1) {
  for (i in genes) {
    x1 <- exp.mat[i, 1]
    y1 <- exp.mat[i, 2]
    plot <- plot + annotate("text", x = x1 + adj.x.t, y = y1 + adj.y.t, 
                            label = i, size = text.size)
    plot <- plot + annotate("segment", x = x1 + adj.x.s, xend = x1, y = y1 + 
                              adj.y.s, yend = y1, size = segment.size)
  }
  return(plot)
}

LabelUR <- function(plot, genes, exp.mat, adj.u.t = 0.1, adj.r.t = 0.15, adj.u.s = 0.05, 
                    adj.r.s = 0.05, ...) {
  return(LabelPoint(plot, genes, exp.mat, adj.y.t = adj.u.t, adj.x.t = adj.r.t, 
                    adj.y.s = adj.u.s, adj.x.s = adj.r.s, ...))
}

LabelUL <- function(plot, genes, exp.mat, adj.u.t = 0.1, adj.l.t = 0.15, adj.u.s = 0.05, 
                    adj.l.s = 0.05, ...) {
  return(LabelPoint(plot, genes, exp.mat, adj.y.t = adj.u.t, adj.x.t = -adj.l.t, 
                    adj.y.s = adj.u.s, adj.x.s = -adj.l.s, ...))
}
# Now, let's see how naive T cells respond to stimulation with interferon gamma.

t.cells <- SubsetData(immune.combined, ident.use = "CD4 Naive T", subset.raw = T)
t.cells <- SetAllIdent(t.cells, id = "stim")
avg.t.cells <- log1p(AverageExpression(t.cells, show.progress = FALSE))
avg.t.cells$gene <- rownames(avg.t.cells)

p1 <- ggplot(avg.t.cells, aes(CTRL, STIM)) + geom_point() + ggtitle("CD4 Naive T Cells")
plot(p1)
# Let's also see how CD14 monocytes respond to stimulation with interferon gamma.

cd14.mono <- SubsetData(immune.combined, ident.use = "CD14 Mono", subset.raw = T)
cd14.mono <- SetAllIdent(cd14.mono, id = "stim")
avg.cd14.mono <- log1p(AverageExpression(cd14.mono, show.progress = FALSE))
avg.cd14.mono$gene <- rownames(avg.cd14.mono)

p2 <- ggplot(avg.cd14.mono, aes(CTRL, STIM)) + geom_point() + ggtitle("CD14 Monocytes")

plot(p2)
# If you were to go in and find the genes that lie above the linear correlation, you would find these:

genes.to.label1 = c("ISG15", "LY6E", "IFI6", "ISG20", "MX1")
genes.to.label2 = c("IFIT2", "IFIT1")
genes.to.label3 = c("CXCL10", "CCL8")

# Now, plot them on the scatterplots and see where they fall.  These end up being the genes that respond specifically to interferon gamma stimulation, both in naive T cells and CD14 monocytes - an interferon gamma response pathway that many cells have.

p1 <- LabelUR(p1, genes = c(genes.to.label1, genes.to.label2), avg.t.cells, 
    adj.u.t = 0.3, adj.u.s = 0.23)
p1 <- LabelUL(p1, genes = genes.to.label3, avg.t.cells, adj.u.t = 0.5, adj.u.s = 0.4, 
    adj.l.t = 0.25, adj.l.s = 0.25)
p2 <- LabelUR(p2, genes = c(genes.to.label1, genes.to.label3), avg.cd14.mono, 
    adj.u.t = 0.3, adj.u.s = 0.23)
p2 <- LabelUL(p2, genes = genes.to.label2, avg.cd14.mono, adj.u.t = 0.5, adj.u.s = 0.4, 
    adj.l.t = 0.25, adj.l.s = 0.25)
plot_grid(p1,p2)
# Now, look for responses specific to particular cell types.

# code to grab particular cell types
immune.combined@meta.data$celltype.stim <- paste0(immune.combined@ident, "_", 
    immune.combined@meta.data$stim)
immune.combined <- StashIdent(immune.combined, save.name = "celltype")
immune.combined <- SetAllIdent(immune.combined, id = "celltype.stim")


# pDC
pDC.interferon.response <- FindMarkers(immune.combined, ident.1 = "pDC_STIM", ident.2 = "pDC_CTRL", print.bar = FALSE)
head(pDC.interferon.response, 15)

# Eryth
Eryth.interferon.response <- FindMarkers(immune.combined, ident.1 = "Eryth_STIM", ident.2 = "Eryth_CTRL", print.bar = FALSE)
head(Eryth.interferon.response, 15)

# Mk
Mk.interferon.response <- FindMarkers(immune.combined, ident.1 = "Mk_STIM", ident.2 = "Mk_CTRL", print.bar = FALSE)
head(Mk.interferon.response, 15)

# DC
DC.interferon.response <- FindMarkers(immune.combined, ident.1 = "DC_STIM", ident.2 = "DC_CTRL", print.bar = FALSE)
head(DC.interferon.response, 15)

# CD14 monocytes
CD14mono.interferon.response <- FindMarkers(immune.combined, ident.1 = "CD14 Mono_STIM", ident.2 = "CD14 Mono_CTRL", print.bar = FALSE)
head(CD14mono.interferon.response, 15)

# CD16 monocytes
CD16mono.interferon.response <- FindMarkers(immune.combined, ident.1 = "CD16 Mono_STIM", ident.2 = "CD16 Mono_CTRL", print.bar = FALSE)
head(CD16mono.interferon.response, 15)

# B activated
Bactiv.interferon.response <- FindMarkers(immune.combined, ident.1 = "B activated_STIM", ident.2 = "B activated_CTRL", print.bar = FALSE)
head(Bactiv.interferon.response, 15)

# B cells
b.interferon.response <- FindMarkers(immune.combined, ident.1 = "B_STIM", ident.2 = "B_CTRL", 
    print.bar = FALSE)
head(b.interferon.response, 15)

# CD8 T cells
CD8T.interferon.response <- FindMarkers(immune.combined, ident.1 = "CD8 T_STIM", ident.2 = "CD8 T_CTRL", 
    print.bar = FALSE)
head(CD8T.interferon.response, 15)

# NK cells (cluster 7)
NK.interferon.response <- FindMarkers(immune.combined, ident.1 = "NK_STIM", ident.2 = "NK_CTRL", 
                                     print.bar = FALSE)
head(NK.interferon.response, 15)

# T activated cells
Tactiv.interferon.response <- FindMarkers(immune.combined, ident.1 = "T activated_STIM", ident.2 = "T activated_CTRL", 
    print.bar = FALSE)
head(Tactiv.interferon.response, 15)

# CD4 naive T cells
CD4naiveT.interferon.response <- FindMarkers(immune.combined, ident.1 = "CD4 Naive T_STIM", ident.2 = "CD4 Naive T_CTRL", print.bar = FALSE)
head(CD4naiveT.interferon.response, 15)

# CD4 Memory T cells
CD4MemT.interferon.response <- FindMarkers(immune.combined, ident.1 = "CD4 Memory T_STIM", ident.2 = "CD4 Memory T_CTRL", print.bar = FALSE)
head(CD4MemT.interferon.response, 15)


# p_val = p-value, 
# avg_logFC = average log fold change, 
# pct.1 = percentage of cells where the gene is detected in the first group, 
# pct.2 = percentage of cells where the gene is detected in the second group
# p_val_adj = adjusted p-value

all_genes<-c(rownames(pDC.interferon.response), rownames(Eryth.interferon.response), rownames(Mk.interferon.response), rownames(DC.interferon.response), rownames(CD14mono.interferon.response), rownames(CD16mono.interferon.response), rownames(Bactiv.interferon.response), rownames(b.interferon.response), rownames(CD8T.interferon.response), rownames(NK.interferon.response), rownames(Tactiv.interferon.response), rownames(CD4naiveT.interferon.response), rownames(CD4MemT.interferon.response))

cell_type_index<-c(length(rownames(pDC.interferon.response)), length(rownames(Eryth.interferon.response), rownames(Mk.interferon.response)), length(rownames(DC.interferon.response)), length(rownames(CD14mono.interferon.response)), length(rownames(CD16mono.interferon.response)), length(rownames(Bactiv.interferon.response)), length(rownames(b.interferon.response)), length(rownames(CD8T.interferon.response)), length(rownames(NK.interferon.response)), length(rownames(Tactiv.interferon.response)), length(rownames(CD4naiveT.interferon.response)), length(rownames(CD4MemT.interferon.response)))

unique_response_genes <- which(table(all_genes)==1)
# Now visualize the top 8 fold changes for these 3 groups of cells.  Are these unique responses?  How would we find cell-type specific responses?

# FeatureHeatmap

# CD14 monocytes
FeatureHeatmap(immune.combined, features.plot = rownames(CD14mono.interferon.response)[1:8], group.by = "stim", pt.size = 0.25, key.position = "top", max.exp = 3)

# CD4 naive T cells
FeatureHeatmap(immune.combined, features.plot = rownames(CD4naiveT.interferon.response)[1:8], group.by = "stim", pt.size = 0.25, key.position = "top", max.exp = 3)

# B cells
FeatureHeatmap(immune.combined, features.plot = rownames(b.interferon.response)[1:8], group.by = "stim", pt.size = 0.25, key.position = "top", max.exp = 3)

Save the objects

# save this work and come back to it later

saveRDS(pbmc, file = paste(directory, "/pbmc_combined.rds", sep = ""))
saveRDS(stim, file = paste(directory, "/stim_combined.rds", sep = ""))
saveRDS(immune.combined, file = paste(directory, "/immune_combined.rds", sep = ""))
LS0tCnRpdGxlOiAiQ1FTIERpc2NvdmVyeSBPcml0ZW50ZWQgRGF0YSBTY2llbmNlIC0gc2NSTkEtc2VxIGRhdGEgYW5hbHlzaXMiCm91dHB1dDogaHRtbF9ub3RlYm9vawotLS0KCiMgSW50cm8KIyMjIyMgQXV0aG9yOiBQYWlnZSAoU3BlbmNlcikgVmVnYSwgVmFuZGVyYmlsdCBVbml2ZXJzaXR5LCBLZW4gTGF1IGxhYgojIyMjIyBUaGlzIHR1dG9yaWFsIHdhcyBtb2RpZmllZCBmcm9tIFNldXJhdCdzIHR1dG9yaWFscyBhdmFpbGFibGUgYXQgdGhlIGxpbmtzIGJlbG93OgojIyMjIyBodHRwczovL3NhdGlqYWxhYi5vcmcvc2V1cmF0L3BibWMza190dXRvcmlhbC5odG1sCiMjIyMjIGh0dHBzOi8vc2F0aWphbGFiLm9yZy9zZXVyYXQvaW1tdW5lX2FsaWdubWVudC5odG1sCiMjIyMjIFlvdSBjYW4gYWxzbyBsZWFybiBtb3JlIGFib3V0IHRoZSBTZXVyYXQgdG9vbCBieSByZWFkaW5nIHRoZWlyIHBhcGVyOgojIyMjIyBodHRwczovL3d3dy5uYXR1cmUuY29tL2FydGljbGVzL25idC40MDk2CgoKIyBEYXRhIGluZm9ybWF0aW9uCiMjIyMjIEthbmcgZXQgYWwuLCAyMDE3IChodHRwczovL3d3dy5uYXR1cmUuY29tL2FydGljbGVzL25idC40MDQyKQojIyMjIyB0byBkb3dubG9hZCBpdCwgZm9sbG93IHRoaXMgbGluazoKIyMjIyMgaHR0cHM6Ly93d3cuZHJvcGJveC5jb20vcy83OXE2ZHR0Zzh5bDIwemcvaW1tdW5lX2FsaWdubWVudF9leHByZXNzaW9uX21hdHJpY2VzLnppcD9kbD0xCiMjIyMjIFRoZSBmaXJzdCBkYXRhc2V0IGlzIDEzLDAxOSBQQk1DcyAocGVyaXBoZXJhbCBibG9vZCBtb25vbnVjbGVhciBjZWxscykuICBUaGUgc2Vjb25kIGlzIDEyLDg3NSBQQk1DcyBzdGltdWxhdGVkIHdpdGggaW50ZXJmZXJvbiBnYW1tYS4KCgojIExlYXJuaW5nIGdvYWxzIGZvciBzZWN0aW9uIDEKIyMjIyMgICAxLiAgQ2hlY2sgdGhlIHF1YWxpdHkgb2YgdGhlIGRhdGEgYW5kIGZpbHRlciBpdC4KIyMjIyMgICAyLiAgTm9ybWFsaXplLCBzY2FsZSwgYW5kIGZpbmQgaGlnaGx5IHZhcmlhYmxlIGdlbmVzIChIVkdzKS4KIyMjIyMgIDMuICBEaW1lbnNpb24gcmVkdWN0aW9uLgojIyMjIyAgIDQuICBEaWZmZXJlbnRpYWwgZ2VuZSBleHByZXNzaW9uLgoKCgoKIyBDYWxsIG9uIHRoZSBTZXVyYXQgZnVuY3Rpb24KCmBgYHtyfQojIGxvYWQgaW4gdGhlIFNldXJhdCBwYWNrYWdlCmxpYnJhcnkoU2V1cmF0KQoKIyBzZXQgeW91ciB3b3JraW5nIGRpcmVjdG9yeQpkaXJlY3Rvcnk8LXNldHdkKCIvVXNlcnMvcGFpZ2V2ZWdhL0Rlc2t0b3AvQ1FTXzIwMThfRGlzY292ZXJ5T3JpZW50ZWREYXRhU2NpZW5jZSIpCgpgYGAKCgojIExvYWQgaW4geW91ciBkYXRhIGFuZCBjcmVhdGUgU2V1cmF0IG9iamVjdAoKYGBge3J9CiMgbG9hZCBpbiB0aGUgZGF0YQogICMgMzUsNjM1IHJvd3MgKGdlbmVzKQogICMgMTMsMDE5IGNlbGxzIChuYW1lcyBhcmUgYmFyY29kZXMpCgpwYm1jLmRhdGEgPC0gcmVhZC50YWJsZShwYXN0ZShkaXJlY3RvcnksIi9pbW11bmVfY29udHJvbF9leHByZXNzaW9uX21hdHJpeC50eHQuZ3oiLCBzZXAgPSAiIiksIHNlcCA9ICJcdCIpCgojIENyZWF0ZSB0aGUgU2V1cmF0IG9iamVjdCB3aXRoIHRoZSByYXcgKG5vbi1ub3JtYWxpemVkIGRhdGEpLiAgS2VlcCBhbGwgZ2VuZXMgZXhwcmVzc2VkIGluID49IDUgY2VsbHMuIAoKcGJtYyA8LSBDcmVhdGVTZXVyYXRPYmplY3QocmF3LmRhdGEgPSBwYm1jLmRhdGEsIG1pbi5jZWxscyA9IDUsIHByb2plY3QgPSAiY29udHJvbF9QQk1DIikKCiMgcGJtY0ByYXcuZGF0YQoKYGBgCgoKIyBGaWx0ZXJpbmcKCmBgYHtyfQojIFRoZSBudW1iZXIgb2YgZ2VuZXMgYW5kIFVNSXMgKG5HZW5lIGFuZCBuVU1JKSBhcmUgYXV0b21hdGljYWxseSBjYWxjdWxhdGVkCiMgZm9yIGV2ZXJ5IG9iamVjdCBieSBTZXVyYXQuIFVzZSB2aW9saW4gcGxvdHMgdG8gdmlzdWFsaXplIG51bWJlciBnZW5lcyBhbmQgbnVtYmVyIFVNSXMuCgpWbG5QbG90KG9iamVjdCA9IHBibWMsIGZlYXR1cmVzLnBsb3QgPSBjKCJuR2VuZSIsICJuVU1JIiksIG5Db2wgPSAyLCBwb2ludC5zaXplLnVzZSA9IDAuMDEpCgojIE5vdGU6IGZpbmRpbmcgdGhlIHBlcmNlbnQgbWl0b2Nob25kcmlhbCBnZW5lcyBmb3IgYSBkYXRhc2V0IGlzIGEgY29tbW9uIFFDIG1ldHJpYywgYnV0LCB0aGlzIGRhdGFzZXQgaXMgYWxyZWFkeSBwcmUtcHJvY2Vzc2VkIHRvIHJlbW92ZSBjZWxscyB0aGF0IGhhdmUgbWl0b2Nob25kcmlhbCBnZW5lIGV4cHJlc3Npb24uCmBgYAoKYGBge3J9CiMgR2VuZVBsb3QgaXMgdHlwaWNhbGx5IHVzZWQgdG8gdmlzdWFsaXplIGdlbmUtZ2VuZSByZWxhdGlvbnNoaXBzLCBidXQgY2FuIGJlIHVzZWQgdG8gc2VlIGhvdyBhbnkgdHdvIHZhcmlhYmxlcyBjb3JyZWxhdGUuICBXZSB3aWxsIHVzZSBpdCB0byBzZWUgaG93IG51bWJlciBVTUlzIGNvcnJlbGF0ZSB0byBudW1iZXIgb2YgZ2VuZXMuCgojIGJ5IHRoZSB3YXksIHdoYXQgd291bGQgYmUgImJhZCIgYW5kIHdoYXQgd291bGQgYmUgImdvb2QiPwoKICAgICAgIyBiYWQgPSBsb3cgbkdlbmUgd2l0aCBoaWdoIG5VTUkgPT4gbWVhbnMgdGhlcmUgYXJlIGZldyB0eXBlcyBvZiBnZW5lcyB3aXRoIGhpZ2ggZXhwcmVzc2lvbgogICAgICAjIGdvb2QgPSBsaW5lYXIgY29ycmVsYXRpb24gbkdlbmUgdG8gblVNSSwgd2l0aCBuVU1JIGFib3V0IDMtNVggZ3JlYXRlciB0aGFuIG5HZW5lCgpHZW5lUGxvdChvYmplY3QgPSBwYm1jLCBnZW5lMSA9ICJuVU1JIiwgZ2VuZTIgPSAibkdlbmUiKQoKIyBOb3RlOiBpZiB5b3UncmUgdHJ5aW5nIHRvIHJlbW92ZSBjZWxscyBleHByZXNzaW5nIGhpZ2ggJSBtaXRvY2hvbmRyaWFsIGdlbmVzLCBwbG90IG5VTUkgdG8gJSBtaXRvIGFuZCBRQyBmcm9tIHRoZXJlLgpgYGAKCmBgYHtyfQojIEJhc2VkIG9uIHRoZSB2aW9saW4gcGxvdCwgZmlsdGVyIG91dCBjZWxscyBieSBjaG9vc2luZyBsb3cgYW5kIGhpZ2ggdGhyZXNob2xkcy4KCiMgZ2V0IHJpZCBvZiBjZWxscyB3aXRoIHZlcnkgZmV3IG5HZW5lcyBvciB3YXkgdG9vIG1hbnkgbkdlbmUKcGJtYyA8LSBGaWx0ZXJDZWxscyhvYmplY3QgPSBwYm1jLCBzdWJzZXQubmFtZXMgPSAibkdlbmUiLCBsb3cudGhyZXNob2xkcyA9IDIwMCwgaGlnaC50aHJlc2hvbGRzID0gMTgwMCkKCiMgaWYgeW91IHdlcmUgcmVtb3ZpbmcgY2VsbHMgd2l0aCBoaWdoICUgbWl0byBleHByZXNzaW9uLCByZW1vdmUgdGhlbSBoZXJlIHdpdGggYXBwcm9wcmlhdGUgdGhyZXNob2xkcy4KCiMgcGJtY0ByYXcuZGF0YQpgYGAKCgoKIyBOb3JtYWxpemF0aW9uCgpgYGB7cn0KIyBOb3JtYWxpemUgdGhlIGRhdGEgLSBsb2cgbm9ybWFsaXphdGlvbiBpcyBkZWZhdWx0LgoKcGJtYyA8LSBOb3JtYWxpemVEYXRhKG9iamVjdCA9IHBibWMpCgojIHBibWNAZGF0YQpgYGAKCgojIEZpbmQgaGlnaGx5IHZhcmlhYmxlIGdlbmVzIChIVkdzKQoKYGBge3J9CiMgRmluZCB0aGUgdG9wIDEsMDAwIG1vc3QgdmFyaWFibGUgZ2VuZXMKICAjIENhbGN1bGF0ZXMgYXZlcmFnZSBleHByZXNzaW9uIGFuZCBkaXNwZXJzaW9uLCB1c2luZyB6LXNjb3JlcyB0byBkZXRlcm1pbmUgb3V0bGllcnMsIHdoaWNoICAgICAgICBhY2NvdW50cyBmb3IgdGhlIHJlbGF0aW9uc2hpcCBiZXR3ZWVuIHZhcmlhYmlsaXR5IGFuZCBhdmcgZXhwcmVzc2lvbi4KCnBibWMgPC0gRmluZFZhcmlhYmxlR2VuZXMocGJtYykKCiMgcGJtY0B2YXIuZ2VuZXMgIyBjb250YWlucyB2YXJpYWJsZSBnZW5lcwojIHBibWNAaHZnLmluZm8gICMgY29udGFpbnMgcmVzdWx0cyBvZiBIVkcgYW5hbHlzaXMKYGBgCgojIFNjYWxlIGRhdGEKCmBgYHtyfQojIHJlbW92ZXMgInVuaW50ZXJlc3Rpbmcgc291cmNlcyBvZiB2YXJpYXRpb24iLCBsaWtlIHRlY2huaWNhbCBub2lzZSwgb3IgY2VsbCBjeWNsZQojIGxpbmVhciByZWdyZXNzaW9uIHRvIHByZWRpY3QgZ2VuZSBleHByZXNzaW9uLiBTZWUgP1NjYWxlRGF0YSBmb3IgbW9yZSBpbmZvcm1hdGlvbi4KCnBibWMgPC0gU2NhbGVEYXRhKHBibWMpCgojIHBibWNAc2NhbGUuZGF0YQpgYGAKCiMgRGltZW5zaW9uYWxpdHkgcmVkdWN0aW9uIChQQ0EpCgpgYGB7cn0KIyBSdW4gUENBIHVzaW5nIHRoZSBIVkdzCgpwYm1jIDwtIFJ1blBDQShvYmplY3QgPSBwYm1jLCBwYy5nZW5lcyA9IHBibWNAdmFyLmdlbmVzLCBwY3MuY29tcHV0ZSA9IDMwLCBkby5wcmludCA9IEZBTFNFKQoKIyBwYm1jQGRyJHBjYQoKYGBgCgojIENob29zZSBQQ3MKCmBgYHtyIGZpZy5oZWlnaHQ9NiwgZmlnLndpZHRoPTEwfQojIFByaW5jaXBsZSBjb21wb25lbnRzIChQQ3MpIGNhcHR1cmUgdGhlIHZhcmlhYmlsaXR5IGluIHlvdXIgZGF0YXNldC4gIEFsdGhvdWdoIHdlIHNwZWNpZmllZCAzMCBQQ3MsIHdlIHdhbnQgdG8gdXNlIG9ubHkgdGhlIFBDcyB0aGF0IGNhcHR1cmUgdmFyaWFiaWxpdHkgYW5kIGRpc2NhcmQgdGhlIHJlc3QgZm9yIGRvd25zdHJlYW0gYW5hbHlzZXMuICBZb3UgbG9vayBmb3Igc2F0dXJhdGlvbiAoZmxhdGxpbmUpIGluIHRoZSByZWxhdGlvbnNoaXAgYmV0d2VlbiB0aGUgbnVtYmVyIG9mIHByaW5jaXBsZSBjb21wb25lbnRzIGFuZCB0aGUgcGVyY2VudGFnZSBvZiB0aGUgdmFyaWFuY2UgZXhwbGFpbmVkLgoKIyAzIHdheXMgdG8gbG9vayBhdCBQQ3MuLi4KCiMgMS4gRWxib3cgcGxvdCBvZiBwcmluY2lwYWwgY29tcG9uZW50cwpQQ0VsYm93UGxvdChvYmplY3QgPSBwYm1jLCBudW0ucGMgPSAzMCkKCiMgMi4gSGVhdG1hcCBvZiBpbmRpdmlkdWFsIFBDcwpQQ0hlYXRtYXAob2JqZWN0ID0gcGJtYywgcGMudXNlID0gMToxNSwgY2VsbHMudXNlID0gNTAwLCBkby5iYWxhbmNlZCA9IFRSVUUsIGxhYmVsLmNvbHVtbiA9IEZBTFNFLCB1c2UuZnVsbCA9IEZBTFNFKQoKUENIZWF0bWFwKG9iamVjdCA9IHBibWMsIHBjLnVzZSA9IDE2OjMwLCBjZWxscy51c2UgPSA1MDAsIGRvLmJhbGFuY2VkID0gVFJVRSwgbGFiZWwuY29sdW1uID0gRkFMU0UsIHVzZS5mdWxsID0gRkFMU0UpCgojIDMuIEphY2tzdHJhdyBwYWNrYWdlIHRvIGRldGVybWluZSBzdGF0aXN0aWNhbGx5IHNpZ25pZmljYW50IHByaW5jaXBhbCBjb21wb25lbnRzCnBibWMgPC0gSmFja1N0cmF3KG9iamVjdCA9IHBibWMsIG51bS5wYyA9IDMwLCBudW0ucmVwbGljYXRlID0gMTAwLCBkaXNwbGF5LnByb2dyZXNzID0gVFJVRSkKSmFja1N0cmF3UGxvdChvYmplY3QgPSBwYm1jLCBQQ3MgPSAxOjMwKQoKYGBgCgojIFVzZSBQQ3MgdG8gY2x1c3RlciBhbmQgdmlzdWFsaXplIGNsdXN0ZXJpbmcgd2l0aCB0LVNORQoKYGBge3J9CiMgQ2x1c3RlcmluZyB1c2luZyBTZXVyYXQncyBtZXRob2QuIFNlZSBtb3JlIGluZm8gaW4gP0ZpbmRDbHVzdGVycy4KICAjIHNwZWNpZnkgdGhlIG51bWJlciBvZiBQQ3MgdXNpbmcgZGltcy51c2UKCnBibWMgPC0gRmluZENsdXN0ZXJzKG9iamVjdCA9IHBibWMsIHJlZHVjdGlvbi50eXBlID0gInBjYSIsIGRpbXMudXNlID0gMToyMCwgcmVzb2x1dGlvbiA9IDAuNiwKICAgICAgICAgICAgICAgICAgICAgcHJpbnQub3V0cHV0ID0gMCwgc2F2ZS5TTk4gPSBUUlVFKQojIHBibWNAaWRlbnQKCiMgUnVuIGRpbWVuc2lvbiByZWR1Y3Rpb24gKHQtU05FKSB0byB2aXN1YWxpemUgdGhlIGNsdXN0ZXJpbmcKcGJtYyA8LSBTZXVyYXQ6OlJ1blRTTkUob2JqZWN0ID0gcGJtYywgZGltcy51c2UgPSAxOjIwLCBkby5mYXN0ID0gVFJVRSkKClRTTkVQbG90KG9iamVjdCA9IHBibWMpCgpgYGAKCgojIERpZmZlcmVudGlhbCBnZW5lIGV4cHJlc3Npb24gLSBvbmUgY2x1c3RlciBhdCBhIHRpbWUKCmBgYHtyfQojIGNsdXN0ZXIgMSB2cyBhbGwgb3RoZXIgY2x1c3RlcnMgcG9vbGVkIHRvZ2V0aGVyCmNsdXN0ZXIxLm1hcmtlcnMgPC0gRmluZE1hcmtlcnMob2JqZWN0ID0gcGJtYywgaWRlbnQuMSA9IDEsIG1pbi5wY3QgPSAwLjI1KQpwcmludCh4ID0gaGVhZCh4ID0gY2x1c3RlcjEubWFya2VycywgbiA9NSkpCgojIGNsdXN0ZXIgMSB2cyBjbHVzdGVyIDUKY2x1c3RlcjEubWFya2Vycy41IDwtIEZpbmRNYXJrZXJzKG9iamVjdCA9IHBibWMsIGlkZW50LjEgPSAxLCBpZGVudC4yID0gNSwgbWluLnBjdCA9IDAuMjUpCnByaW50KHggPSBoZWFkKHggPSBjbHVzdGVyMS5tYXJrZXJzLjUsIG4gPTUpKQoKCmBgYAoKIyBEaWZmZXJlbnRpYWwgZ2VuZSBleHByZXNzaW9uIC0gaXRlcmF0ZSB0aHJvdWdoIGFsbCBjbHVzdGVycwoKYGBge3J9CiMgRmluZCBtYXJrZXJzIGZvciBldmVyeSBjbHVzdGVyIGNvbXBhcmVkIHRvIGFsbCByZW1haW5pbmcgY2VsbHMsIHJlcG9ydCBvbmx5IHBvc2l0aXZlIG9uZXMKcGJtYy5tYXJrZXJzIDwtIEZpbmRBbGxNYXJrZXJzKG9iamVjdCA9IHBibWMsIG9ubHkucG9zID0gVFJVRSwgbWluLnBjdCA9IDAuMjUsIHRocmVzaC51c2UgPSAwLjI1KQoKCiMgV2hhdCBhYm91dCB1c2luZyBhIGRpZmZlcmVudCB0ZXN0PyAgT24geW91ciBvd24gdGltZSwgdHJ5aW5nIGFkZGluZyB0aGUgYXJndW1lbnQgdGVzdC51c2UgPSAicm9jIiBhbmQgc2VlIGhvdyBkaWZmZXJlbnQgeW91ciByZXN1bHRzIGFyZS4gCmBgYAoKCiMgVmlzdWFsaXplIGRpZmZlcmVudGlhbGx5IGV4cHJlc3NlZCBnZW5lcwoKYGBge3J9CiMgVGhlIGxpc3Qgb2YgREUgZ2VuZXMgaXMgbG9uZyBhbmQgaXQgd291bGQgYmUgdG9vIG11Y2ggaW5mb3JtYXRpb24gdG8gdmlzdWFsaXplIHRoZW0gYWxsLiBTbywgbGV0J3MgcGxvdCB0aGUgdG9wIDUgZmVhdHVyZXMgKGdlbmVzKSBmb3IgZWFjaCBjbHVzdGVyLiAgCgojIGNhbGwgbGlicmFyeSBkcGx5ciB0byB1c2Ugc29tZSBvZiBpdHMgZnVuY3Rpb25zCmxpYnJhcnkoZHBseXIpCiMgZ3JvdXAgYnkgY2x1c3RlciB1c2luZyBncm91cF9ieSBmdW5jdGlvbiBhbmQgc2hvdyB0b3AgNSByZXN1bHRzIHVzaW5nIHRvcF9uIGZ1bmN0aW9uCnBibWMubWFya2Vycy5ncm91cGVkIDwtIGdyb3VwX2J5KHBibWMubWFya2VycywgY2x1c3RlcikKcGJtYy5tYXJrZXJzLmdyb3VwZWQudG9wIDwtIHRvcF9uKHBibWMubWFya2Vycy5ncm91cGVkLCA1LCBhdmdfbG9nRkMpCiMgY29udmVydCBpdCB0byBhIG1hdHJpeCBmb3IgZG93bnN0cmVhbSBhcHBsaWNhdGlvbnMgKHRydXN0IG1lKQpwYm1jLm1hcmtlcnMuZ3JvdXBlZC50b3AgPC0gYXMubWF0cml4KHBibWMubWFya2Vycy5ncm91cGVkLnRvcCkKcHJpbnQocGJtYy5tYXJrZXJzLmdyb3VwZWQudG9wKQoKIyBGaXJzdCwgaXNvbGF0ZSB0aGUgdG9wIDUgREUgZ2VuZXMgZm9yIGVhY2ggY2x1c3Rlci4KY2x1c3Rlcl8wIDwtIHdoaWNoKHBibWMubWFya2Vycy5ncm91cGVkLnRvcFssImNsdXN0ZXIiXSA9PSAwKQpjbHVzdGVyXzEgPC0gd2hpY2gocGJtYy5tYXJrZXJzLmdyb3VwZWQudG9wWywiY2x1c3RlciJdID09IDEpCmNsdXN0ZXJfMiA8LSB3aGljaChwYm1jLm1hcmtlcnMuZ3JvdXBlZC50b3BbLCJjbHVzdGVyIl0gPT0gMikKY2x1c3Rlcl8zIDwtIHdoaWNoKHBibWMubWFya2Vycy5ncm91cGVkLnRvcFssImNsdXN0ZXIiXSA9PSAzKQpjbHVzdGVyXzQgPC0gd2hpY2gocGJtYy5tYXJrZXJzLmdyb3VwZWQudG9wWywiY2x1c3RlciJdID09IDQpCmNsdXN0ZXJfNSA8LSB3aGljaChwYm1jLm1hcmtlcnMuZ3JvdXBlZC50b3BbLCJjbHVzdGVyIl0gPT0gNSkKY2x1c3Rlcl82IDwtIHdoaWNoKHBibWMubWFya2Vycy5ncm91cGVkLnRvcFssImNsdXN0ZXIiXSA9PSA2KQpjbHVzdGVyXzcgPC0gd2hpY2gocGJtYy5tYXJrZXJzLmdyb3VwZWQudG9wWywiY2x1c3RlciJdID09IDcpCmNsdXN0ZXJfOCA8LSB3aGljaChwYm1jLm1hcmtlcnMuZ3JvdXBlZC50b3BbLCJjbHVzdGVyIl0gPT0gOCkKY2x1c3Rlcl85IDwtIHdoaWNoKHBibWMubWFya2Vycy5ncm91cGVkLnRvcFssImNsdXN0ZXIiXSA9PSA5KQpjbHVzdGVyXzEwIDwtIHdoaWNoKHBibWMubWFya2Vycy5ncm91cGVkLnRvcFssImNsdXN0ZXIiXSA9PSAxMCkKCgojIDMgb3B0aW9ucyBmb3IgdmlzdWFsaXphdGlvbi4uLgoKIyAxLiBWaW9saW4gcGxvdHMKIyBpZiB5b3Ugd2FudCB0byBzZWUgdGhlIHBvaW50cywgaW5zZXJ0IHBvaW50LnNpemUudXNlID0gMC4wMSBpbnRvIHRoZSBhcmd1bWVudCBsaXN0CgojICMgc3BlY2lmeSBjbHVzdGVyIDAgaGVyZQpWbG5QbG90KG9iamVjdCA9IHBibWMsIGZlYXR1cmVzLnBsb3QgPSBwYm1jLm1hcmtlcnMuZ3JvdXBlZC50b3BbY2x1c3Rlcl8wLCAiZ2VuZSJdLAogICAgICAgIHBvaW50LnNpemUudXNlID0gMCkKIyAjIHNwZWNpZnkgY2x1c3RlciAxIGhlcmUKVmxuUGxvdChvYmplY3QgPSBwYm1jLCBmZWF0dXJlcy5wbG90ID0gcGJtYy5tYXJrZXJzLmdyb3VwZWQudG9wW2NsdXN0ZXJfMSwgImdlbmUiXSwKICAgICAgICBwb2ludC5zaXplLnVzZSA9IDApCiMgIyBzcGVjaWZ5IGNsdXN0ZXIgMiBoZXJlClZsblBsb3Qob2JqZWN0ID0gcGJtYywgZmVhdHVyZXMucGxvdCA9IHBibWMubWFya2Vycy5ncm91cGVkLnRvcFtjbHVzdGVyXzIsICJnZW5lIl0sCiAgICAgICAgcG9pbnQuc2l6ZS51c2UgPSAwKQojICMgc3BlY2lmeSBjbHVzdGVyIDMgaGVyZQpWbG5QbG90KG9iamVjdCA9IHBibWMsIGZlYXR1cmVzLnBsb3QgPSBwYm1jLm1hcmtlcnMuZ3JvdXBlZC50b3BbY2x1c3Rlcl8zLCAiZ2VuZSJdLAogICAgICAgIHBvaW50LnNpemUudXNlID0gMCkKIyAjIHNwZWNpZnkgY2x1c3RlciA0IGhlcmUKVmxuUGxvdChvYmplY3QgPSBwYm1jLCBmZWF0dXJlcy5wbG90ID0gcGJtYy5tYXJrZXJzLmdyb3VwZWQudG9wW2NsdXN0ZXJfNCwgImdlbmUiXSwKICAgICAgICBwb2ludC5zaXplLnVzZSA9IDApCiMgIyBzcGVjaWZ5IGNsdXN0ZXIgNSBoZXJlClZsblBsb3Qob2JqZWN0ID0gcGJtYywgZmVhdHVyZXMucGxvdCA9IHBibWMubWFya2Vycy5ncm91cGVkLnRvcFtjbHVzdGVyXzUsICJnZW5lIl0sCiAgICAgICAgcG9pbnQuc2l6ZS51c2UgPSAwKQojICMgc3BlY2lmeSBjbHVzdGVyIDYgaGVyZQpWbG5QbG90KG9iamVjdCA9IHBibWMsIGZlYXR1cmVzLnBsb3QgPSBwYm1jLm1hcmtlcnMuZ3JvdXBlZC50b3BbY2x1c3Rlcl82LCAiZ2VuZSJdLAogICAgICAgIHBvaW50LnNpemUudXNlID0gMCkKIyAjIHNwZWNpZnkgY2x1c3RlciA3IGhlcmUKVmxuUGxvdChvYmplY3QgPSBwYm1jLCBmZWF0dXJlcy5wbG90ID0gcGJtYy5tYXJrZXJzLmdyb3VwZWQudG9wW2NsdXN0ZXJfNywgImdlbmUiXSwKICAgICAgICBwb2ludC5zaXplLnVzZSA9IDApCiMgIyBzcGVjaWZ5IGNsdXN0ZXIgOCBoZXJlClZsblBsb3Qob2JqZWN0ID0gcGJtYywgZmVhdHVyZXMucGxvdCA9IHBibWMubWFya2Vycy5ncm91cGVkLnRvcFtjbHVzdGVyXzgsICJnZW5lIl0sCiAgICAgICAgcG9pbnQuc2l6ZS51c2UgPSAwKQojICMgc3BlY2lmeSBjbHVzdGVyIDkgaGVyZQpWbG5QbG90KG9iamVjdCA9IHBibWMsIGZlYXR1cmVzLnBsb3QgPSBwYm1jLm1hcmtlcnMuZ3JvdXBlZC50b3BbY2x1c3Rlcl85LCAiZ2VuZSJdLAogICAgICAgIHBvaW50LnNpemUudXNlID0gMCkKIyAjIHNwZWNpZnkgY2x1c3RlciAxMCBoZXJlClZsblBsb3Qob2JqZWN0ID0gcGJtYywgZmVhdHVyZXMucGxvdCA9IHBibWMubWFya2Vycy5ncm91cGVkLnRvcFtjbHVzdGVyXzEwLCAiZ2VuZSJdLAogICAgICAgIHBvaW50LnNpemUudXNlID0gMCkKCgoKIyAyLiBGZWF0dXJlIHBsb3QKIyBZb3UgY291bGQgY2hhbmdlIHRoZSByZWR1Y3Rpb24udXNlIHRvIGRvIFBDQSBvciBJQ0EgaW5zdGVhZCBvZiB0LVNORSwgYnV0IHNpbmNlIHdlIGtub3cgd2hlcmUgb3VyIGNsdXN0ZXJzIHdlcmUgbWFwcGVkIGluIHRoZSBjb250ZXh0IG9mIHQtU05FLCB0aGlzIG1ha2VzIHRoZSBtb3N0IHNlbnNlIGZvciB1cy4KCiMgQ2x1c3RlciAwCkZlYXR1cmVQbG90KG9iamVjdCA9IHBibWMsIGZlYXR1cmVzLnBsb3QgPSBwYm1jLm1hcmtlcnMuZ3JvdXBlZC50b3BbY2x1c3Rlcl8wLCAiZ2VuZSJdLCBjb2xzLnVzZSA9IGMoImdyZXkiLCAiYmx1ZSIpLCByZWR1Y3Rpb24udXNlID0gInRzbmUiKQojIENsdXN0ZXIgMQpGZWF0dXJlUGxvdChvYmplY3QgPSBwYm1jLCBmZWF0dXJlcy5wbG90ID0gcGJtYy5tYXJrZXJzLmdyb3VwZWQudG9wW2NsdXN0ZXJfMSwgImdlbmUiXSwgY29scy51c2UgPSBjKCJncmV5IiwgImJsdWUiKSwgcmVkdWN0aW9uLnVzZSA9ICJ0c25lIikKIyBDbHVzdGVyIDIKRmVhdHVyZVBsb3Qob2JqZWN0ID0gcGJtYywgZmVhdHVyZXMucGxvdCA9IHBibWMubWFya2Vycy5ncm91cGVkLnRvcFtjbHVzdGVyXzIsICJnZW5lIl0sIGNvbHMudXNlID0gYygiZ3JleSIsICJibHVlIiksIHJlZHVjdGlvbi51c2UgPSAidHNuZSIpCiMgQ2x1c3RlciAzCkZlYXR1cmVQbG90KG9iamVjdCA9IHBibWMsIGZlYXR1cmVzLnBsb3QgPSBwYm1jLm1hcmtlcnMuZ3JvdXBlZC50b3BbY2x1c3Rlcl8zLCAiZ2VuZSJdLCBjb2xzLnVzZSA9IGMoImdyZXkiLCAiYmx1ZSIpLCByZWR1Y3Rpb24udXNlID0gInRzbmUiKQojIENsdXN0ZXIgNApGZWF0dXJlUGxvdChvYmplY3QgPSBwYm1jLCBmZWF0dXJlcy5wbG90ID0gcGJtYy5tYXJrZXJzLmdyb3VwZWQudG9wW2NsdXN0ZXJfNCwgImdlbmUiXSwgY29scy51c2UgPSBjKCJncmV5IiwgImJsdWUiKSwgcmVkdWN0aW9uLnVzZSA9ICJ0c25lIikKIyBDbHVzdGVyIDUKRmVhdHVyZVBsb3Qob2JqZWN0ID0gcGJtYywgZmVhdHVyZXMucGxvdCA9IHBibWMubWFya2Vycy5ncm91cGVkLnRvcFtjbHVzdGVyXzUsICJnZW5lIl0sIGNvbHMudXNlID0gYygiZ3JleSIsICJibHVlIiksIHJlZHVjdGlvbi51c2UgPSAidHNuZSIpCiMgQ2x1c3RlciA2CkZlYXR1cmVQbG90KG9iamVjdCA9IHBibWMsIGZlYXR1cmVzLnBsb3QgPSBwYm1jLm1hcmtlcnMuZ3JvdXBlZC50b3BbY2x1c3Rlcl82LCAiZ2VuZSJdLCBjb2xzLnVzZSA9IGMoImdyZXkiLCAiYmx1ZSIpLCByZWR1Y3Rpb24udXNlID0gInRzbmUiKQojIENsdXN0ZXIgNwpGZWF0dXJlUGxvdChvYmplY3QgPSBwYm1jLCBmZWF0dXJlcy5wbG90ID0gcGJtYy5tYXJrZXJzLmdyb3VwZWQudG9wW2NsdXN0ZXJfNywgImdlbmUiXSwgY29scy51c2UgPSBjKCJncmV5IiwgImJsdWUiKSwgcmVkdWN0aW9uLnVzZSA9ICJ0c25lIikKIyBDbHVzdGVyIDgKRmVhdHVyZVBsb3Qob2JqZWN0ID0gcGJtYywgZmVhdHVyZXMucGxvdCA9IHBibWMubWFya2Vycy5ncm91cGVkLnRvcFtjbHVzdGVyXzgsICJnZW5lIl0sIGNvbHMudXNlID0gYygiZ3JleSIsICJibHVlIiksIHJlZHVjdGlvbi51c2UgPSAidHNuZSIpCiMgQ2x1c3RlciA5CkZlYXR1cmVQbG90KG9iamVjdCA9IHBibWMsIGZlYXR1cmVzLnBsb3QgPSBwYm1jLm1hcmtlcnMuZ3JvdXBlZC50b3BbY2x1c3Rlcl85LCAiZ2VuZSJdLCBjb2xzLnVzZSA9IGMoImdyZXkiLCAiYmx1ZSIpLCByZWR1Y3Rpb24udXNlID0gInRzbmUiKQojIENsdXN0ZXIgMTAKRmVhdHVyZVBsb3Qob2JqZWN0ID0gcGJtYywgZmVhdHVyZXMucGxvdCA9IHBibWMubWFya2Vycy5ncm91cGVkLnRvcFtjbHVzdGVyXzEwLCAiZ2VuZSJdLCBjb2xzLnVzZSA9IGMoImdyZXkiLCAiYmx1ZSIpLCByZWR1Y3Rpb24udXNlID0gInRzbmUiKQoKCiMgMy4gSGVhdG1hcAojIEhlYXRtYXBzIGhhdmUgZW5vdWdoIHJvb20gdG8gc2hvdyB5b3UgdGhlIHRvcCA1IChvciBtb3JlISkgZ2VuZXMgZm9yIGVhY2ggY2x1c3Rlci4gIExldCdzIGRvIHRoZSB0b3AgNSwgYnV0IGlmIHlvdSBoYXZlIHRpbWUsIHRyeSBjaGFuZ2luZyB0aGUgdG9wIGZlYXR1cmVzIHRvIDEwIG9yIG1vcmUuCgpEb0hlYXRtYXAob2JqZWN0ID0gcGJtYywgZ2VuZXMudXNlID0gcGJtYy5tYXJrZXJzLmdyb3VwZWQudG9wWyAsICJnZW5lIl0sIHNsaW0uY29sLmxhYmVsID0gVFJVRSwgcmVtb3ZlLmtleSA9IFRSVUUpCgpgYGAKCiMgU2F2ZSB0aGUgb2JqZWN0IGFuZCB3b3Jrc3BhY2UKCmBgYHtyfQojIHNhdmUgdGhpcyBvYmplY3QgYW5kIGNvbWUgYmFjayB0byBpdCBsYXRlcgpzYXZlUkRTKHBibWMsIGZpbGUgPSBwYXN0ZShkaXJlY3RvcnksICIvcGJtY19vbmx5LnJkcyIsIHNlcCA9ICIiKSkKCiMgc2F2ZSB3b3JrIGluIFIKc2F2ZS5pbWFnZShwYXN0ZShkaXJlY3RvcnksICIvc2VjdGlvbjEuUkRhdGEiLCBzZXAgPSAiIikpCgpgYGAKCnNhdmUuaW1hZ2UocGFzdGUoZGlyZWN0b3J5LCAiL3NlY3Rpb24xLlJEYXRhIiwgc2VwID0gIiIpKQoKCgojIE5vdyBsZXQncyBhZGQgaW4gYSBzZWNvbmQgZGF0YXNldCwgd2hpY2ggaXMgUEJNQ3Mgc3RpbXVsYXRlZCB3aXRoIGludGVyZmVyb24gZ2FtbWEKCmBgYHtyfQojIGxvYWQgaW4gdGhlIGRhdGEgLSBob3cgbWFueSBjZWxscyBhbmQgaG93IG1hbnkgZ2VuZXM/ICB1c2UgZGltKHN0aW0uZGF0YSkgdG8gZmluZCBvdXQKc3RpbS5kYXRhIDwtIHJlYWQudGFibGUocGFzdGUoZGlyZWN0b3J5LCAiL2ltbXVuZV9zdGltdWxhdGVkX2V4cHJlc3Npb25fbWF0cml4LnR4dC5neiIsIHNlcCA9ICIiKSwgc2VwID0gIlx0IikKCiMgY3JlYXRlIHRoZSBTZXVyYXQgb2JqZWN0IGZvciB0aGUgc3RpbXVsYXRlZCBQQk1DcwpzdGltIDwtIENyZWF0ZVNldXJhdE9iamVjdChyYXcuZGF0YSA9IHN0aW0uZGF0YSwgbWluLmNlbGxzID0gNSwgcHJvamVjdCA9ICJzdGltX1BCTUMiKQoKIyBBbHNvIHJlLWNyZWF0ZSB0aGUgY29udHJvbCBQQk1DIG9iamVjdCBiZWNhdXNlIHdlIHdpbGwgY2hhbmdlIGEgY291cGxlIHRoaW5ncyB0aGlzIHRpbWUgYW5kIHdlIGRvbid0IHdhbnQgdG8gaGF2ZSB0aGUgcHJldmlvdXMgaW5mbyBzdG9yZWQgaW4gdGhlIFNldXJhdCBvYmplY3QuCnBibWMgPC0gQ3JlYXRlU2V1cmF0T2JqZWN0KHJhdy5kYXRhID0gcGJtYy5kYXRhLCBtaW4uY2VsbHMgPSA1LCBwcm9qZWN0ID0gImNvbnRyb2xfUEJNQyIpCgpgYGAKCiMjIyMgV2UgaGF2ZSBhIGZldyBnb2FscyBpbiB0aGlzIHNlY3Rpb24gb2YgdGhlIGFuYWx5c2lzLiAgCiMjIyMjICAgMS4gIGlkZW50aWZ5IGNlbGwgdHlwZXMgcHJlc2VudCBpbiBib3RoIHRoZSBjb250cm9sIHBibWMgYW5kIHN0aW11bGF0ZWQgcGJtYyBkYXRhc2V0cwojIyMjIyAgIDIuICBmaW5kIG1hcmtlcnMgdGhhdCBpZGVudGlmeSBjZWxsIHR5cGVzIHRoYXQgYXJlIGNvbnNlcnZlZCBpbiBjb250cm9sIGFuZCBzdGltdWxhdGVkIHBibWNzCiMjIyMjICAgMy4gIGNvbXBhcmUgdGhlIGRhdGFzZXRzIHRvIGZpbmQgY2VsbC10eXBlIHNwZWNpZmljIHJlc3BvbnNlcyB0byBpbnRlcmZlcm9uIHN0aW11bGF0aW9uIAoKIyMjIyBSZWFsbHksIHRoZSBnb2FsIGlzIHRvIHJlaW5mb3JjZSB0aGUgbWFqb3Igc3RlcHMgaW4gc2NSTkEtc2VxIGFuYWx5c2lzIGJ5IGdvaW5nIG92ZXIgc29tZSBvZiB0aGVtIGFnYWluLiAgVGhpcyB0aW1lLCB3ZSBzaG93IGhvdyBldmVuIGEgc2ltcGxlIGV4cGVyaW1lbnRhbCBkZXNpZ24gKGNvbnRyb2wgdnMgc3RpbXVsYXRlZCkgY2FuIGNvbXBsaWNhdGUgdGhlIGFuYWx5c2lzLgoKCgojIEZpbHRlcmluZywgbm9ybWFsaXphdGlvbiwgc2NhbGluZywgYW5kIGZpbmRpbmcgSFZHcwpgYGB7cn0KClZsblBsb3Qob2JqZWN0ID0gc3RpbSwgZmVhdHVyZXMucGxvdCA9IGMoIm5HZW5lIiwgIm5VTUkiKSwgbkNvbCA9IDIsIHBvaW50LnNpemUudXNlID0gMC4wMDEpCgpgYGBgCgpgYGB7cn0KR2VuZVBsb3Qob2JqZWN0ID0gc3RpbSwgZ2VuZTEgPSAiblVNSSIsIGdlbmUyID0gIm5HZW5lIikKYGBgCgpgYGB7cn0KIyBGaWx0ZXIgb3V0IHRoZSBzdGltdWxhdGVkIHBibWNzIHdpdGggbG93IHRvdGFsIGdlbmUgZXhwcmVzc2lvbgogICMgZm9yIHRoaXMgYW5hbHlzaXMsIHdlIHdhbnQgb25seSB0aGUgZ2VuZXMgd2l0aCBhdCBsZWFzdCBhdmVyYWdlIG5HZW5lCnN0aW0gPC0gRmlsdGVyQ2VsbHMob2JqZWN0ID0gc3RpbSwgc3Vic2V0Lm5hbWVzID0gIm5HZW5lIiwgbG93LnRocmVzaG9sZHMgPSA1MDAsIGhpZ2gudGhyZXNob2xkcyA9IDE4MDApCiMgbm9ybWFsaXplCnN0aW0gPC0gTm9ybWFsaXplRGF0YShzdGltKQojIHNjYWxlCnN0aW0gPC0gU2NhbGVEYXRhKHN0aW0pCiMgRmluZCB2YXJpYWJsZSBnZW5lcwpzdGltIDwtRmluZFZhcmlhYmxlR2VuZXMoc3RpbSkKIyBtYWtlIG1ldGFkYXRhIGluZGljYXRpbmcgaXQgaXMgc3RpbXVsYXRlZApzdGltQG1ldGEuZGF0YSRzdGltIDwtICJTVElNIgoKIyB5ZXMsIHdlIGFscmVhZHkgZGlkIHRoaXMgZm9yIHRoZSBjb250cm9sIGRhdGFzZXQsIGJ1dCB0aGUgZG93bnN0cmVhbSBhbmFseXNpcyByZXF1aXJlcyBhIGJpdCBtb3JlIHNlbGVjdGl2ZSBmaWx0ZXJpbmcgZm9yIGl0IHRvIHJ1bi4gIFRoZSBvbmx5IGNoYW5nZXMgd2UgYXJlIG1ha2luZyBpcyBjaG9vc2luZyBjZWxscyB3aXRoIDUwMC0xODAwIGdlbmVzIGluc3RlYWQgb2YgMjAwLTE4MDAgZ2VuZXMuCnBibWMgPC0gRmlsdGVyQ2VsbHMob2JqZWN0ID0gcGJtYywgc3Vic2V0Lm5hbWVzID0gIm5HZW5lIiwgbG93LnRocmVzaG9sZHMgPSA1MDAsIGhpZ2gudGhyZXNob2xkcyA9IDE4MDApCiMgbm9ybWFsaXplCnBibWMgPC0gTm9ybWFsaXplRGF0YShwYm1jKQojIHNjYWxlCnBibWMgPC0gU2NhbGVEYXRhKHBibWMpCiMgRmluZCB2YXJpYWJsZSBnZW5lcwpwYm1jIDwtRmluZFZhcmlhYmxlR2VuZXMocGJtYykKIyBtYWtlIG1ldGFkYXRhIGluZGljYXRpbmcgaXQgaXMgc3RpbXVsYXRlZApwYm1jQG1ldGEuZGF0YSRzdGltIDwtICJDVFJMIgoKCmBgYAoKYGBge3J9CiMgTm93IGZpbmQgdGhlIGhpZ2hseSB2YXJpYWJsZSBnZW5lcyB0aGF0IGFyZSBzaGFyZWQgYmV0d2VlbiB0aGUgdHdvIGRhdGFzZXRzIChjb250cm9sIGFuZCBzdGltdWxhdGVkKQoKZy4xIDwtIGhlYWQocm93bmFtZXMocGJtY0BodmcuaW5mbyksIDEwMDApCmcuMiA8LSBoZWFkKHJvd25hbWVzKHN0aW1AaHZnLmluZm8pLCAxMDAwKQoKIyBrZWVwcyBvbmx5IHVuaXF1ZSBnZW5lbmFtZXMgKGdldHMgcmlkIG9mIGR1cGxpY2F0ZXMpCmdlbmVzLnVzZSA8LSB1bmlxdWUoYyhnLjEsIGcuMikpCmdlbmVzLnVzZSA8LSBpbnRlcnNlY3QoZ2VuZXMudXNlLCByb3duYW1lcyhwYm1jQHNjYWxlLmRhdGEpKQpnZW5lcy51c2UgPC0gaW50ZXJzZWN0KGdlbmVzLnVzZSwgcm93bmFtZXMoc3RpbUBzY2FsZS5kYXRhKSkKYGBgCgoKIyBDYW5vbmljYWwgQ29ycmVsYXRpb24gQW5hbHlzaXMgKENDQSkKCmBgYHtyfQojIFJ1biBDQ0EgdG8gaWRlbnRpZnkgY29tbW9uIHNvdXJjZXMgb2YgdmFyaWF0aW9uIGJldHdlZW4gdGhlIHR3byBkYXRhc2V0cwoKIyBDQ0EgZmluZHMgbGluZWFyIGNvbWJpbmF0aW9ucyBvZiBmZWF0dXJlcyBhY3Jvc3MgdGhlIGNvbmRpdGlvbnMgdGhhdCBhcmUgbWF4aW1hbGx5ICAgICAgICAgICBjb3JyZWxhdGVkLiAgSW4gc2hvcnQsIGl0IHNlYXJjaGVzIGZvciBwYXR0ZXJucyBpbiB0aGUgZGF0YXNldHMgdGhhdCBhcmUgY29tbW9uLgoKaW1tdW5lLmNvbWJpbmVkIDwtIFJ1bkNDQShwYm1jLCBzdGltLCBnZW5lcy51c2UgPSBnZW5lcy51c2UsIG51bS5jYyA9IDMwKQoKIyBWaXN1YWxpemUgQ0NBIHJlc3VsdHMKCiAgIyBwbG90IENDMSB2cyBDQzIKcDEgPC0gRGltUGxvdChvYmplY3QgPSBpbW11bmUuY29tYmluZWQsIHJlZHVjdGlvbi51c2UgPSAiY2NhIiwgZ3JvdXAuYnkgPSAic3RpbSIsIHB0LnNpemUgPSAwLjUsIGRvLnJldHVybiA9IFRSVUUpCiAgIyB2aW9saW4gcGxvdCB0byB2aXN1YWxpemUgQ0MxIGFuZCBDQzIKcDIgPC0gVmxuUGxvdChvYmplY3QgPSBpbW11bmUuY29tYmluZWQsIGZlYXR1cmVzLnBsb3QgPSAiQ0MxIiwgZ3JvdXAuYnkgPSAic3RpbSIsIGRvLnJldHVybiA9IFRSVUUsIHBvaW50LnNpemUudXNlID0gMC4wMDEpCnAzIDwtIFZsblBsb3Qob2JqZWN0ID0gaW1tdW5lLmNvbWJpbmVkLCBmZWF0dXJlcy5wbG90ID0gIkNDMiIsIGdyb3VwLmJ5ID0gInN0aW0iLCBkby5yZXR1cm4gPSBUUlVFLCBwb2ludC5zaXplLnVzZSA9IDAuMDAxKQpwbG90KHAxKQpwbG90X2dyaWQocDIsIHAzKQoKYGBgCgojIENob29zZSBDQ3MKCmBgYHtyfQoKIyBNdWNoIGxpa2Ugc2VsZWN0aW5nIHRoZSBudW1iZXIgb2YgUENzIHRvIHVzZSBpbiB0aGUgZmlyc3QgYW5hbHlzaXMsIHdlIG11c3Qgc2VsZWN0IHRoZSBudW1iZXIgb2YgQ0NzIHRvIHVzZSBoZXJlLiAgVGhlIE1ldGFnZW5CaWNvclBsb3QgZnVuY3Rpb24gYmVsb3cgaXMgYW5hbG9nb3VzIHRvIHRoZSBlbGJvdyBwbG90IG1ldGhvZCB1c2VkIGluIHRoZSBmaXJzdCBhbmFseXNpcyBvZiAxIGRhdGFzZXQuICBZb3UgbG9vayBmb3Igc2F0dXJhdGlvbiAoZmxhdGxpbmUpLgoKIyAyIHdheXMgdG8gcGljayBDQ3MsIGJ1dCB3ZSB3b24ndCBydW4gdGhlIGZpcnN0IGJlY2F1c2UgaXQgdGFrZXMgdG9vIGxvbmcgdG8gY29tcHV0ZSBmb3IgdGhlIHRpbWUgd2UgaGF2ZSBpbiBjbGFzcy4gIEZlZWwgZnJlZSB0byB0cnkgaXQgb24geW91ciBvd24hCgojICMgMS4gIE1ldGFnZW5lQmljb3JQbG90CnA0IDwtIE1ldGFnZW5lQmljb3JQbG90KGltbXVuZS5jb21iaW5lZCwgZ3JvdXBpbmcudmFyID0gInN0aW0iLCBkaW1zLmV2YWwgPSAxOjMwLCBkaXNwbGF5LnByb2dyZXNzID0gVFJVRSkKCiMgMi4gSGVhdG1hcCAtIHVzZSB0aGUgdG9wIDUwMCBjZWxscwogICMgeW91IG5lZWQgdGhyZWUgcGFnZXMgdG8gc2VlIGFsbCB0aGUgQ0NzCiAgIyBwdXJwbGUgPSBsb3cgZXhwcmVzc2lvbgogICMgeWVsbG93ID0gaGlnaApEaW1IZWF0bWFwKG9iamVjdCA9IGltbXVuZS5jb21iaW5lZCwgcmVkdWN0aW9uLnR5cGUgPSAiY2NhIiwgY2VsbHMudXNlID0gNTAwLCBkaW0udXNlID0gMTo5LAogICAgICAgICAgIGRvLmJhbGFuY2VkID0gVFJVRSkKCkRpbUhlYXRtYXAob2JqZWN0ID0gaW1tdW5lLmNvbWJpbmVkLCByZWR1Y3Rpb24udHlwZSA9ICJjY2EiLCBjZWxscy51c2UgPSA1MDAsIGRpbS51c2UgPSAxMDoxOCwKICAgICAgICAgICBkby5iYWxhbmNlZCA9IFRSVUUpCgpEaW1IZWF0bWFwKG9iamVjdCA9IGltbXVuZS5jb21iaW5lZCwgcmVkdWN0aW9uLnR5cGUgPSAiY2NhIiwgY2VsbHMudXNlID0gNTAwLCBkaW0udXNlID0gMTk6MzAsCiAgICAgICAgICAgZG8uYmFsYW5jZWQgPSBUUlVFKQoKIyBIb3cgbWFueSBDQ3MgZG8geW91IHRoaW5rIHdlIHNob3VsZCBwaWNrPwoKYGBgCgojIEFsaWduIHRoZSBDQ0Egc3Vic3BhY2VzIGFuZCB2aXN1YWxpemUKCmBgYHtyfQojIHVzZSB0aGUgZmlyc3QgMjAgQ0NzIHRvIGdldCBhIG5ldyBkaW1lbnNpb24gcmVkdWN0aW9uIHRoYXQgeW91IGNhbiB2aXN1YWxpemUgYW5kIHVzZSBmb3IgY2x1c3RlcmluZwoKaW1tdW5lLmNvbWJpbmVkIDwtIEFsaWduU3Vic3BhY2UoaW1tdW5lLmNvbWJpbmVkLCByZWR1Y3Rpb24udHlwZSA9ICJjY2EiLCBncm91cGluZy52YXIgPSAic3RpbSIsIAogICAgICAgICAgICAgICAgICAgIGRpbXMuYWxpZ24gPSAxOjIwKQoKIyB2aXN1YWxpemUgdGhlIGFsaWduZWQgQ0NBIC0gaG93IGRvZXMgdGhpcyBjb21wYXJlIHRvIGJlZm9yZSBhbGlnbm1lbnQ/CgpwMSA8LSBWbG5QbG90KG9iamVjdCA9IGltbXVuZS5jb21iaW5lZCwgZmVhdHVyZXMucGxvdCA9ICJBQ0MxIiwgZ3JvdXAuYnkgPSAic3RpbSIsIGRvLnJldHVybiA9ICJUUlVFIiwgcG9pbnQuc2l6ZS51c2UgPSAwLjAxKQpwMiA8LSBWbG5QbG90KG9iamVjdCA9IGltbXVuZS5jb21iaW5lZCwgZmVhdHVyZXMucGxvdCA9ICJBQ0MyIiwgZ3JvdXAuYnkgPSAic3RpbSIsIGRvLnJldHVybiA9ICJUUlVFIiwgcG9pbnQuc2l6ZS51c2UgPSAwLjAxKQpwbG90X2dyaWQocDEsIHAyKQoKIyBvYmplY3RAZHIkcmVkdWN0aW9uLnR5cGUuYWxpZ25lZAoKYGBgCgojIEZpbmQgY2x1c3RlcnMgdXNpbmcgdGhlIGFsaWduZWQgZGF0YQoKYGBge3J9CiMgUnVuIHQtU05FCmltbXVuZS5jb21iaW5lZCA8LSBTZXVyYXQ6OlJ1blRTTkUoaW1tdW5lLmNvbWJpbmVkLCByZWR1Y3Rpb24udXNlID0gImNjYS5hbGlnbmVkIiwgZGltcy51c2UgPSAxOjIwLCBkby5mYXN0ID0gVCkKCiMgY2x1c3RlciBiYXNlZCBvbiB0LVNORSBzcGFjZQppbW11bmUuY29tYmluZWQgPC0gRmluZENsdXN0ZXJzKGltbXVuZS5jb21iaW5lZCwgcmVkdWN0aW9uLnR5cGUgPSAiY2NhLmFsaWduZWQiLCByZXNvbHV0aW9uID0gMC42LCBkaW1zLnVzZSA9IDE6MjApCgojIFZpc3VhbGl6ZSB0aGUgZGF0YXNldCBhbGlnbm1lbnQgYW5kIGNvbWJpbmVkIGNsdXN0ZXJpbmcgcmVzdWx0cyBvbiB0LVNORQpwMSA8LSBUU05FUGxvdChpbW11bmUuY29tYmluZWQsIGRvLnJldHVybiA9IFQsIHB0LnNpemUgPSAwLjUsIGdyb3VwLmJ5ID0gInN0aW0iKQpwMiA8LSBUU05FUGxvdChpbW11bmUuY29tYmluZWQsIGRvLmxhYmVsID0gVCwgIGRvLnJldHVybiA9IFQsIHB0LnNpemUgPSAwLjUpCnBsb3RfZ3JpZChwMSwgcDIpCgpgYGAKCgojIERpZmZlcmVudGlhbCBnZW5lIGV4cHJlc3Npb24gb2YgbWFya2VycyBjb25zZXJ2ZWQgYWNyb3NzIGNvbmRpdGlvbnMKCmBgYHtyfQoKIyBJZGVudGlmeSBjb25zZXJ2ZWQgY2VsbCB0eXBlIG1hcmtlcnMgCiAgIyBDb25zZXJ2ZWQgY2VsbCB0eXBlIG1ha3JlcyBhcmUgdGhvc2UgdGhhdCBpZGVudGlmeSBhIHBhcnRpY3VsYXIgY2VsbCB0eXBlIGluIGJvdGggY29udHJvbCAgICAgICBwYm1jIGFuZCBzdGltdWxhdGVkIHBibWMpLiAgVGhlIEZpbmRDb25zZXJ2ZWRtYXJrZXJzIGZ1bmN0aW9uIHVzZXMgZGlmZmVyZW50aWFsIGdlbmUgICAgICAgICAgICAgZXhwcmVzc2lvbiBhbmFseXNpcyB0byBkbyB0aGlzLgoKIyBBcyBhbiBleGFtcGxlLCBsZXQncyBsb29rIGZvciBtYXJrZXJzIHRoYXQgYXJlIGNvbnNlcnZlZCBiZXR3ZWVuIGNvbnRyb2wgYW5kIHN0aW11bGF0ZWQgY2VsbHMgICAgaW4gY2x1c3RlciA3LCBkZWZpbmVkIGluIHRoZSAiaWRlbnQuMSIgYXJndW1lbnQuCmNsdXN0ZXI3Lm1hcmtlcnMgPC0gRmluZENvbnNlcnZlZE1hcmtlcnMoaW1tdW5lLmNvbWJpbmVkLCBpZGVudC4xID0gNywgZ3JvdXBpbmcudmFyID0gInN0aW0iLCBwcmludC5iYXIgPSBGQUxTRSkKIyB2aWV3IHRoZSBmaXJzdCBmZXcgbWFya2VycwpjbHVzdGVyNy5tYXJrZXJzWzE6OSxdCgojIENvbmZpcm0gdGhhdCB0aG9zZSBtYXJrZXJzIGZvciBjbHVzdGVyIDcgaWRlbnRpZmllZCBmYWxsIGludG8gdGhhdCBsb2NhdGlvbiBvbiB0LVNORS4KRmVhdHVyZVBsb3Qob2JqZWN0ID0gaW1tdW5lLmNvbWJpbmVkLCBmZWF0dXJlcy5wbG90ID0gcm93bmFtZXMoY2x1c3RlcjcubWFya2VycylbMTo5XSwgbWluLmN1dG9mZiA9ICJxOSIsIGNvbHMudXNlID0gYygibGlnaHRncmV5IiwgImJsdWUiKSwgcHQuc2l6ZSA9IDAuNSkKCiMgQmFzZWQgb24gdGhvc2UgcmVzdWx0cywgd2hhdCBjZWxsIHR5cGUgaXMgY2x1c3RlciA3PwoKCmBgYAoKCgpgYGB7cn0KIyBETyBOT1QgUlVOIFRISVMgTU9SRSBUSEFOIE9OQ0UKCiMgRXhwbG9yZSBtYXJrZXJzIG9mIGRpZmZlcmVudCBpbW11bmUgY2VsbCBwb3B1bGF0aW9ucyB0byB0cnkgdG8gaWRlbnRpZnkgd2hhdCBjZWxsIHR5cGUgZWFjaCBjbHVzdGVyIGNvbnRhaW5zLiAgU2VsZWN0IGNhbmRpZGF0ZXMgYmFzZWQgb2ZmIG9mIHByaW9yIGtub3dsZWRnZSBvZiBpbW11bmUgY2VsbCBtYXJrZXJzLgpGZWF0dXJlUGxvdChvYmplY3QgPSBpbW11bmUuY29tYmluZWQsIGZlYXR1cmVzLnBsb3QgPSBjKCJDRDNEIiwgIlNFTEwiLCAiQ1JFTSIsICJDRDhBIiwgIkdOTFkiLCAiQ0Q3OUEiLCAiRkNHUjNBIiwgIkNDTDIiLCAiUFBCUCIpLCBtaW4uY3V0b2ZmID0gInE5IiwgY29scy51c2UgPSBjKCJsaWdodGdyZXkiLCAiYmx1ZSIpLCBwdC5zaXplID0gMC41KQoKIyBOb3csIHlvdSBjYW4gcmVuYW1lIHlvdXIgY2x1c3RlcnMgYXMgdGhlc2UgY2VsbCBuYW1lcy4gIFRydXN0IHRoYXQgdGhleSBhcmUgY29ycmVjdCA6KQpuZXcuaWRlbnQgPC0gYygiQ0QxNCBNb25vIiwgIkNENCBOYWl2ZSBUIiwgIkNENCBNZW1vcnkgVCIsICJCIiwgIkNEMTYgTW9ubyIsCiAgICAiVCBhY3RpdmF0ZWQiLCAiQ0Q4IFQiLCAiTksiLCAiREMiLCAiQiBhY3RpdmF0ZWQiLCAiTWsiLCAicERDIiwgIkVyeXRoIikKZm9yIChpIGluIDA6MTIpIHsKICAgIGltbXVuZS5jb21iaW5lZCA8LSBSZW5hbWVJZGVudChvYmplY3QgPSBpbW11bmUuY29tYmluZWQsIG9sZC5pZGVudC5uYW1lID0gaSwKICAgICAgICBuZXcuaWRlbnQubmFtZSA9IG5ldy5pZGVudFtpICsgMV0pCn0KCiMgUnVuIHQtU05FIHRvIHNlZSB5b3VyIG5ldyBjbHVzdGVyIG5hbWVzClRTTkVQbG90KGltbXVuZS5jb21iaW5lZCwgZG8ubGFiZWwgPSBULCBwdC5zaXplID0gMC41KQoKdXNlIFNQbGl0RG90UGxvdEdHIHRvIHBsb3QgbWFya2VycyBvZiBlYWNoIGNlbGwgdHlwZSAodXNpbmcgcHJpb3Iga25vd2xlZGdlIGFnYWluKS4KaW1tdW5lLmNvbWJpbmVkQGlkZW50IDwtIGZhY3RvcihpbW11bmUuY29tYmluZWRAaWRlbnQsIGxldmVscyA9IChjKCJwREMiLCAiRXJ5dGgiLAogICAgIk1rIiwgIkRDIiwgIkNEMTQgTW9ubyIsICJDRDE2IE1vbm8iLCAiQiBhY3RpdmF0ZWQiLCAiQiIsICJDRDggVCIsICJOSyIsCiAgICAiVCBhY3RpdmF0ZWQiLCAiQ0Q0IE5haXZlIFQiLCAiQ0Q0IE1lbW9yeSBUIikpKQptYXJrZXJzLnRvLnBsb3QgPC0gYygiQ0QzRCIsICJDUkVNIiwgIkhTUEgxIiwgIlNFTEwiLCAiR0lNQVA1IiwgIkNBQ1lCUCIsICJHTkxZIiwKICAgICJOS0c3IiwgIkNDTDUiLCAiQ0Q4QSIsICJNUzRBMSIsICJDRDc5QSIsICJNSVIxNTVIRyIsICJOTUUxIiwgIkZDR1IzQSIsCiAgICAiVk1PMSIsICJDQ0wyIiwgIlMxMDBBOSIsICJITEEtRFFBMSIsICJHUFIxODMiLCAiUFBCUCIsICJHTkcxMSIsICJIQkEyIiwKICAgICJIQkIiLCAiVFNQQU4xMyIsICJJTDNSQSIsICJJR0oiKQpzZHAgPC0gU3BsaXREb3RQbG90R0coaW1tdW5lLmNvbWJpbmVkLCBnZW5lcy5wbG90ID0gcmV2KG1hcmtlcnMudG8ucGxvdCksIGNvbHMudXNlID0gYygiYmx1ZSIsIAogICAgInJlZCIpLCB4LmxhYi5yb3QgPSBULCBwbG90LmxlZ2VuZCA9IFQsIGRvdC5zY2FsZSA9IDMsIGRvLnJldHVybiA9IFQsIGdyb3VwaW5nLnZhciA9ICJzdGltIikKYGBgYAoKCiMgRGlmZmVyZW50aWFsIGdlbmUgZXhwcmVzc2lvbiBvZiBtYXJrZXJzIHRoYXQgY2hhbmdlIGFjcm9zcyBjb25kaXRpb25zIChjdHJsIHZzIHN0aW11bGF0ZWQpLgoKYGBge3J9CiMgSXQncyBncmVhdCB0aGF0IHdlIGNhbiBpZGVudGlmeSBjb25zZXJ2ZWQgbWFya2VycyBhY3Jvc3MgZGF0YXNldHMgYmVjYXVzZSBpdCBhbGxvd3MgdXMgdG8gYmUgY29uZmlkZW50IHRoYXQgd2UgaGF2ZSBpZGVudGlmaWVkIHN1YnBvcHVsYXRpb25zIG9mIGNlbGwgdHlwZXMgdGhhdCBhcmUgcHJlc2VudCBpbiBib3RoIGRhdGFzZXRzLCBjdHJsIHZzIHN0aW11bGF0ZWQuICBIb3dldmVyLCB3aGF0IGlzIHJlYWxseSBpbnRlcmVzdGluZyBpcyB3aGF0IENIQU5HRVMgd2hlbiB5b3Ugc3RpbXVsYXRlIHRoZSBjZWxscy4gIExldCdzIGZpbmQgb3V0LgoKIyBGaXJzdCB3ZSBuZWVkIHRvIGRlZmluZSBzb21lIHBsb3R0aW5nIGZ1bmN0aW9ucyB0byBtYWtlIGxhYmVsaW5nIGVhc2llci4gIERvbid0IHdvcnJ5IGFib3V0IHRyeWluZyB0byB1bmRlcnN0YW5kIHdoYXQgZWFjaCBsaW5lIGlzIGRvaW5nLCBpdCBpcyBub3QgaW50ZXJlc3RpbmcuICBKdXN0IHJ1biBpdCEKCkxhYmVsUG9pbnQgPC0gZnVuY3Rpb24ocGxvdCwgZ2VuZXMsIGV4cC5tYXQsIGFkai54LnQgPSAwLCBhZGoueS50ID0gMCwgYWRqLngucyA9IDAsIAogICAgICAgICAgICAgICAgICAgICAgIGFkai55LnMgPSAwLCB0ZXh0LnNpemUgPSAyLjUsIHNlZ21lbnQuc2l6ZSA9IDAuMSkgewogIGZvciAoaSBpbiBnZW5lcykgewogICAgeDEgPC0gZXhwLm1hdFtpLCAxXQogICAgeTEgPC0gZXhwLm1hdFtpLCAyXQogICAgcGxvdCA8LSBwbG90ICsgYW5ub3RhdGUoInRleHQiLCB4ID0geDEgKyBhZGoueC50LCB5ID0geTEgKyBhZGoueS50LCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgIGxhYmVsID0gaSwgc2l6ZSA9IHRleHQuc2l6ZSkKICAgIHBsb3QgPC0gcGxvdCArIGFubm90YXRlKCJzZWdtZW50IiwgeCA9IHgxICsgYWRqLngucywgeGVuZCA9IHgxLCB5ID0geTEgKyAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgYWRqLnkucywgeWVuZCA9IHkxLCBzaXplID0gc2VnbWVudC5zaXplKQogIH0KICByZXR1cm4ocGxvdCkKfQoKTGFiZWxVUiA8LSBmdW5jdGlvbihwbG90LCBnZW5lcywgZXhwLm1hdCwgYWRqLnUudCA9IDAuMSwgYWRqLnIudCA9IDAuMTUsIGFkai51LnMgPSAwLjA1LCAKICAgICAgICAgICAgICAgICAgICBhZGouci5zID0gMC4wNSwgLi4uKSB7CiAgcmV0dXJuKExhYmVsUG9pbnQocGxvdCwgZ2VuZXMsIGV4cC5tYXQsIGFkai55LnQgPSBhZGoudS50LCBhZGoueC50ID0gYWRqLnIudCwgCiAgICAgICAgICAgICAgICAgICAgYWRqLnkucyA9IGFkai51LnMsIGFkai54LnMgPSBhZGouci5zLCAuLi4pKQp9CgpMYWJlbFVMIDwtIGZ1bmN0aW9uKHBsb3QsIGdlbmVzLCBleHAubWF0LCBhZGoudS50ID0gMC4xLCBhZGoubC50ID0gMC4xNSwgYWRqLnUucyA9IDAuMDUsIAogICAgICAgICAgICAgICAgICAgIGFkai5sLnMgPSAwLjA1LCAuLi4pIHsKICByZXR1cm4oTGFiZWxQb2ludChwbG90LCBnZW5lcywgZXhwLm1hdCwgYWRqLnkudCA9IGFkai51LnQsIGFkai54LnQgPSAtYWRqLmwudCwgCiAgICAgICAgICAgICAgICAgICAgYWRqLnkucyA9IGFkai51LnMsIGFkai54LnMgPSAtYWRqLmwucywgLi4uKSkKfQoKYGBgCgpgYGB7cn0KIyBOb3csIGxldCdzIHNlZSBob3cgbmFpdmUgVCBjZWxscyByZXNwb25kIHRvIHN0aW11bGF0aW9uIHdpdGggaW50ZXJmZXJvbiBnYW1tYS4KCnQuY2VsbHMgPC0gU3Vic2V0RGF0YShpbW11bmUuY29tYmluZWQsIGlkZW50LnVzZSA9ICJDRDQgTmFpdmUgVCIsIHN1YnNldC5yYXcgPSBUKQp0LmNlbGxzIDwtIFNldEFsbElkZW50KHQuY2VsbHMsIGlkID0gInN0aW0iKQphdmcudC5jZWxscyA8LSBsb2cxcChBdmVyYWdlRXhwcmVzc2lvbih0LmNlbGxzLCBzaG93LnByb2dyZXNzID0gRkFMU0UpKQphdmcudC5jZWxscyRnZW5lIDwtIHJvd25hbWVzKGF2Zy50LmNlbGxzKQoKcDEgPC0gZ2dwbG90KGF2Zy50LmNlbGxzLCBhZXMoQ1RSTCwgU1RJTSkpICsgZ2VvbV9wb2ludCgpICsgZ2d0aXRsZSgiQ0Q0IE5haXZlIFQgQ2VsbHMiKQpwbG90KHAxKQpgYGAKCmBgYHtyfQojIExldCdzIGFsc28gc2VlIGhvdyBDRDE0IG1vbm9jeXRlcyByZXNwb25kIHRvIHN0aW11bGF0aW9uIHdpdGggaW50ZXJmZXJvbiBnYW1tYS4KCmNkMTQubW9ubyA8LSBTdWJzZXREYXRhKGltbXVuZS5jb21iaW5lZCwgaWRlbnQudXNlID0gIkNEMTQgTW9ubyIsIHN1YnNldC5yYXcgPSBUKQpjZDE0Lm1vbm8gPC0gU2V0QWxsSWRlbnQoY2QxNC5tb25vLCBpZCA9ICJzdGltIikKYXZnLmNkMTQubW9ubyA8LSBsb2cxcChBdmVyYWdlRXhwcmVzc2lvbihjZDE0Lm1vbm8sIHNob3cucHJvZ3Jlc3MgPSBGQUxTRSkpCmF2Zy5jZDE0Lm1vbm8kZ2VuZSA8LSByb3duYW1lcyhhdmcuY2QxNC5tb25vKQoKcDIgPC0gZ2dwbG90KGF2Zy5jZDE0Lm1vbm8sIGFlcyhDVFJMLCBTVElNKSkgKyBnZW9tX3BvaW50KCkgKyBnZ3RpdGxlKCJDRDE0IE1vbm9jeXRlcyIpCgpwbG90KHAyKQpgYGAKCmBgYHtyfQojIElmIHlvdSB3ZXJlIHRvIGdvIGluIGFuZCBmaW5kIHRoZSBnZW5lcyB0aGF0IGxpZSBhYm92ZSB0aGUgbGluZWFyIGNvcnJlbGF0aW9uLCB5b3Ugd291bGQgZmluZCB0aGVzZToKCmdlbmVzLnRvLmxhYmVsMSA9IGMoIklTRzE1IiwgIkxZNkUiLCAiSUZJNiIsICJJU0cyMCIsICJNWDEiKQpnZW5lcy50by5sYWJlbDIgPSBjKCJJRklUMiIsICJJRklUMSIpCmdlbmVzLnRvLmxhYmVsMyA9IGMoIkNYQ0wxMCIsICJDQ0w4IikKCiMgTm93LCBwbG90IHRoZW0gb24gdGhlIHNjYXR0ZXJwbG90cyBhbmQgc2VlIHdoZXJlIHRoZXkgZmFsbC4gIFRoZXNlIGVuZCB1cCBiZWluZyB0aGUgZ2VuZXMgdGhhdCByZXNwb25kIHNwZWNpZmljYWxseSB0byBpbnRlcmZlcm9uIGdhbW1hIHN0aW11bGF0aW9uLCBib3RoIGluIG5haXZlIFQgY2VsbHMgYW5kIENEMTQgbW9ub2N5dGVzIC0gYW4gaW50ZXJmZXJvbiBnYW1tYSByZXNwb25zZSBwYXRod2F5IHRoYXQgbWFueSBjZWxscyBoYXZlLgoKcDEgPC0gTGFiZWxVUihwMSwgZ2VuZXMgPSBjKGdlbmVzLnRvLmxhYmVsMSwgZ2VuZXMudG8ubGFiZWwyKSwgYXZnLnQuY2VsbHMsIAogICAgYWRqLnUudCA9IDAuMywgYWRqLnUucyA9IDAuMjMpCnAxIDwtIExhYmVsVUwocDEsIGdlbmVzID0gZ2VuZXMudG8ubGFiZWwzLCBhdmcudC5jZWxscywgYWRqLnUudCA9IDAuNSwgYWRqLnUucyA9IDAuNCwgCiAgICBhZGoubC50ID0gMC4yNSwgYWRqLmwucyA9IDAuMjUpCnAyIDwtIExhYmVsVVIocDIsIGdlbmVzID0gYyhnZW5lcy50by5sYWJlbDEsIGdlbmVzLnRvLmxhYmVsMyksIGF2Zy5jZDE0Lm1vbm8sIAogICAgYWRqLnUudCA9IDAuMywgYWRqLnUucyA9IDAuMjMpCnAyIDwtIExhYmVsVUwocDIsIGdlbmVzID0gZ2VuZXMudG8ubGFiZWwyLCBhdmcuY2QxNC5tb25vLCBhZGoudS50ID0gMC41LCBhZGoudS5zID0gMC40LCAKICAgIGFkai5sLnQgPSAwLjI1LCBhZGoubC5zID0gMC4yNSkKcGxvdF9ncmlkKHAxLHAyKQpgYGAKCmBgYHtyfQojIE5vdywgbG9vayBmb3IgcmVzcG9uc2VzIHNwZWNpZmljIHRvIHBhcnRpY3VsYXIgY2VsbCB0eXBlcy4KCiMgY29kZSB0byBncmFiIHBhcnRpY3VsYXIgY2VsbCB0eXBlcwppbW11bmUuY29tYmluZWRAbWV0YS5kYXRhJGNlbGx0eXBlLnN0aW0gPC0gcGFzdGUwKGltbXVuZS5jb21iaW5lZEBpZGVudCwgIl8iLCAKICAgIGltbXVuZS5jb21iaW5lZEBtZXRhLmRhdGEkc3RpbSkKaW1tdW5lLmNvbWJpbmVkIDwtIFN0YXNoSWRlbnQoaW1tdW5lLmNvbWJpbmVkLCBzYXZlLm5hbWUgPSAiY2VsbHR5cGUiKQppbW11bmUuY29tYmluZWQgPC0gU2V0QWxsSWRlbnQoaW1tdW5lLmNvbWJpbmVkLCBpZCA9ICJjZWxsdHlwZS5zdGltIikKCgojIHBEQwpwREMuaW50ZXJmZXJvbi5yZXNwb25zZSA8LSBGaW5kTWFya2VycyhpbW11bmUuY29tYmluZWQsIGlkZW50LjEgPSAicERDX1NUSU0iLCBpZGVudC4yID0gInBEQ19DVFJMIiwgcHJpbnQuYmFyID0gRkFMU0UpCmhlYWQocERDLmludGVyZmVyb24ucmVzcG9uc2UsIDE1KQoKIyBFcnl0aApFcnl0aC5pbnRlcmZlcm9uLnJlc3BvbnNlIDwtIEZpbmRNYXJrZXJzKGltbXVuZS5jb21iaW5lZCwgaWRlbnQuMSA9ICJFcnl0aF9TVElNIiwgaWRlbnQuMiA9ICJFcnl0aF9DVFJMIiwgcHJpbnQuYmFyID0gRkFMU0UpCmhlYWQoRXJ5dGguaW50ZXJmZXJvbi5yZXNwb25zZSwgMTUpCgojIE1rCk1rLmludGVyZmVyb24ucmVzcG9uc2UgPC0gRmluZE1hcmtlcnMoaW1tdW5lLmNvbWJpbmVkLCBpZGVudC4xID0gIk1rX1NUSU0iLCBpZGVudC4yID0gIk1rX0NUUkwiLCBwcmludC5iYXIgPSBGQUxTRSkKaGVhZChNay5pbnRlcmZlcm9uLnJlc3BvbnNlLCAxNSkKCiMgREMKREMuaW50ZXJmZXJvbi5yZXNwb25zZSA8LSBGaW5kTWFya2VycyhpbW11bmUuY29tYmluZWQsIGlkZW50LjEgPSAiRENfU1RJTSIsIGlkZW50LjIgPSAiRENfQ1RSTCIsIHByaW50LmJhciA9IEZBTFNFKQpoZWFkKERDLmludGVyZmVyb24ucmVzcG9uc2UsIDE1KQoKIyBDRDE0IG1vbm9jeXRlcwpDRDE0bW9uby5pbnRlcmZlcm9uLnJlc3BvbnNlIDwtIEZpbmRNYXJrZXJzKGltbXVuZS5jb21iaW5lZCwgaWRlbnQuMSA9ICJDRDE0IE1vbm9fU1RJTSIsIGlkZW50LjIgPSAiQ0QxNCBNb25vX0NUUkwiLCBwcmludC5iYXIgPSBGQUxTRSkKaGVhZChDRDE0bW9uby5pbnRlcmZlcm9uLnJlc3BvbnNlLCAxNSkKCiMgQ0QxNiBtb25vY3l0ZXMKQ0QxNm1vbm8uaW50ZXJmZXJvbi5yZXNwb25zZSA8LSBGaW5kTWFya2VycyhpbW11bmUuY29tYmluZWQsIGlkZW50LjEgPSAiQ0QxNiBNb25vX1NUSU0iLCBpZGVudC4yID0gIkNEMTYgTW9ub19DVFJMIiwgcHJpbnQuYmFyID0gRkFMU0UpCmhlYWQoQ0QxNm1vbm8uaW50ZXJmZXJvbi5yZXNwb25zZSwgMTUpCgojIEIgYWN0aXZhdGVkCkJhY3Rpdi5pbnRlcmZlcm9uLnJlc3BvbnNlIDwtIEZpbmRNYXJrZXJzKGltbXVuZS5jb21iaW5lZCwgaWRlbnQuMSA9ICJCIGFjdGl2YXRlZF9TVElNIiwgaWRlbnQuMiA9ICJCIGFjdGl2YXRlZF9DVFJMIiwgcHJpbnQuYmFyID0gRkFMU0UpCmhlYWQoQmFjdGl2LmludGVyZmVyb24ucmVzcG9uc2UsIDE1KQoKIyBCIGNlbGxzCmIuaW50ZXJmZXJvbi5yZXNwb25zZSA8LSBGaW5kTWFya2VycyhpbW11bmUuY29tYmluZWQsIGlkZW50LjEgPSAiQl9TVElNIiwgaWRlbnQuMiA9ICJCX0NUUkwiLCAKICAgIHByaW50LmJhciA9IEZBTFNFKQpoZWFkKGIuaW50ZXJmZXJvbi5yZXNwb25zZSwgMTUpCgojIENEOCBUIGNlbGxzCkNEOFQuaW50ZXJmZXJvbi5yZXNwb25zZSA8LSBGaW5kTWFya2VycyhpbW11bmUuY29tYmluZWQsIGlkZW50LjEgPSAiQ0Q4IFRfU1RJTSIsIGlkZW50LjIgPSAiQ0Q4IFRfQ1RSTCIsIAogICAgcHJpbnQuYmFyID0gRkFMU0UpCmhlYWQoQ0Q4VC5pbnRlcmZlcm9uLnJlc3BvbnNlLCAxNSkKCiMgTksgY2VsbHMgKGNsdXN0ZXIgNykKTksuaW50ZXJmZXJvbi5yZXNwb25zZSA8LSBGaW5kTWFya2VycyhpbW11bmUuY29tYmluZWQsIGlkZW50LjEgPSAiTktfU1RJTSIsIGlkZW50LjIgPSAiTktfQ1RSTCIsIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgcHJpbnQuYmFyID0gRkFMU0UpCmhlYWQoTksuaW50ZXJmZXJvbi5yZXNwb25zZSwgMTUpCgojIFQgYWN0aXZhdGVkIGNlbGxzClRhY3Rpdi5pbnRlcmZlcm9uLnJlc3BvbnNlIDwtIEZpbmRNYXJrZXJzKGltbXVuZS5jb21iaW5lZCwgaWRlbnQuMSA9ICJUIGFjdGl2YXRlZF9TVElNIiwgaWRlbnQuMiA9ICJUIGFjdGl2YXRlZF9DVFJMIiwgCiAgICBwcmludC5iYXIgPSBGQUxTRSkKaGVhZChUYWN0aXYuaW50ZXJmZXJvbi5yZXNwb25zZSwgMTUpCgojIENENCBuYWl2ZSBUIGNlbGxzCkNENG5haXZlVC5pbnRlcmZlcm9uLnJlc3BvbnNlIDwtIEZpbmRNYXJrZXJzKGltbXVuZS5jb21iaW5lZCwgaWRlbnQuMSA9ICJDRDQgTmFpdmUgVF9TVElNIiwgaWRlbnQuMiA9ICJDRDQgTmFpdmUgVF9DVFJMIiwgcHJpbnQuYmFyID0gRkFMU0UpCmhlYWQoQ0Q0bmFpdmVULmludGVyZmVyb24ucmVzcG9uc2UsIDE1KQoKIyBDRDQgTWVtb3J5IFQgY2VsbHMKQ0Q0TWVtVC5pbnRlcmZlcm9uLnJlc3BvbnNlIDwtIEZpbmRNYXJrZXJzKGltbXVuZS5jb21iaW5lZCwgaWRlbnQuMSA9ICJDRDQgTWVtb3J5IFRfU1RJTSIsIGlkZW50LjIgPSAiQ0Q0IE1lbW9yeSBUX0NUUkwiLCBwcmludC5iYXIgPSBGQUxTRSkKaGVhZChDRDRNZW1ULmludGVyZmVyb24ucmVzcG9uc2UsIDE1KQoKCiMgcF92YWwgPSBwLXZhbHVlLCAKIyBhdmdfbG9nRkMgPSBhdmVyYWdlIGxvZyBmb2xkIGNoYW5nZSwgCiMgcGN0LjEgPSBwZXJjZW50YWdlIG9mIGNlbGxzIHdoZXJlIHRoZSBnZW5lIGlzIGRldGVjdGVkIGluIHRoZSBmaXJzdCBncm91cCwgCiMgcGN0LjIgPSBwZXJjZW50YWdlIG9mIGNlbGxzIHdoZXJlIHRoZSBnZW5lIGlzIGRldGVjdGVkIGluIHRoZSBzZWNvbmQgZ3JvdXAKIyBwX3ZhbF9hZGogPSBhZGp1c3RlZCBwLXZhbHVlCmBgYAoKYGBge3J9CgphbGxfZ2VuZXM8LWMocm93bmFtZXMocERDLmludGVyZmVyb24ucmVzcG9uc2UpLCByb3duYW1lcyhFcnl0aC5pbnRlcmZlcm9uLnJlc3BvbnNlKSwgcm93bmFtZXMoTWsuaW50ZXJmZXJvbi5yZXNwb25zZSksIHJvd25hbWVzKERDLmludGVyZmVyb24ucmVzcG9uc2UpLCByb3duYW1lcyhDRDE0bW9uby5pbnRlcmZlcm9uLnJlc3BvbnNlKSwgcm93bmFtZXMoQ0QxNm1vbm8uaW50ZXJmZXJvbi5yZXNwb25zZSksIHJvd25hbWVzKEJhY3Rpdi5pbnRlcmZlcm9uLnJlc3BvbnNlKSwgcm93bmFtZXMoYi5pbnRlcmZlcm9uLnJlc3BvbnNlKSwgcm93bmFtZXMoQ0Q4VC5pbnRlcmZlcm9uLnJlc3BvbnNlKSwgcm93bmFtZXMoTksuaW50ZXJmZXJvbi5yZXNwb25zZSksIHJvd25hbWVzKFRhY3Rpdi5pbnRlcmZlcm9uLnJlc3BvbnNlKSwgcm93bmFtZXMoQ0Q0bmFpdmVULmludGVyZmVyb24ucmVzcG9uc2UpLCByb3duYW1lcyhDRDRNZW1ULmludGVyZmVyb24ucmVzcG9uc2UpKQoKY2VsbF90eXBlX2luZGV4PC1jKGxlbmd0aChyb3duYW1lcyhwREMuaW50ZXJmZXJvbi5yZXNwb25zZSkpLCBsZW5ndGgocm93bmFtZXMoRXJ5dGguaW50ZXJmZXJvbi5yZXNwb25zZSksIHJvd25hbWVzKE1rLmludGVyZmVyb24ucmVzcG9uc2UpKSwgbGVuZ3RoKHJvd25hbWVzKERDLmludGVyZmVyb24ucmVzcG9uc2UpKSwgbGVuZ3RoKHJvd25hbWVzKENEMTRtb25vLmludGVyZmVyb24ucmVzcG9uc2UpKSwgbGVuZ3RoKHJvd25hbWVzKENEMTZtb25vLmludGVyZmVyb24ucmVzcG9uc2UpKSwgbGVuZ3RoKHJvd25hbWVzKEJhY3Rpdi5pbnRlcmZlcm9uLnJlc3BvbnNlKSksIGxlbmd0aChyb3duYW1lcyhiLmludGVyZmVyb24ucmVzcG9uc2UpKSwgbGVuZ3RoKHJvd25hbWVzKENEOFQuaW50ZXJmZXJvbi5yZXNwb25zZSkpLCBsZW5ndGgocm93bmFtZXMoTksuaW50ZXJmZXJvbi5yZXNwb25zZSkpLCBsZW5ndGgocm93bmFtZXMoVGFjdGl2LmludGVyZmVyb24ucmVzcG9uc2UpKSwgbGVuZ3RoKHJvd25hbWVzKENENG5haXZlVC5pbnRlcmZlcm9uLnJlc3BvbnNlKSksIGxlbmd0aChyb3duYW1lcyhDRDRNZW1ULmludGVyZmVyb24ucmVzcG9uc2UpKSkKCnVuaXF1ZV9yZXNwb25zZV9nZW5lcyA8LSB3aGljaCh0YWJsZShhbGxfZ2VuZXMpPT0xKQoKYGBgCgpgYGB7cn0KIyBOb3cgdmlzdWFsaXplIHRoZSB0b3AgOCBmb2xkIGNoYW5nZXMgZm9yIHRoZXNlIDMgZ3JvdXBzIG9mIGNlbGxzLiAgQXJlIHRoZXNlIHVuaXF1ZSByZXNwb25zZXM/ICBIb3cgd291bGQgd2UgZmluZCBjZWxsLXR5cGUgc3BlY2lmaWMgcmVzcG9uc2VzPwoKIyBGZWF0dXJlSGVhdG1hcAoKIyBDRDE0IG1vbm9jeXRlcwpGZWF0dXJlSGVhdG1hcChpbW11bmUuY29tYmluZWQsIGZlYXR1cmVzLnBsb3QgPSByb3duYW1lcyhDRDE0bW9uby5pbnRlcmZlcm9uLnJlc3BvbnNlKVsxOjhdLCBncm91cC5ieSA9ICJzdGltIiwgcHQuc2l6ZSA9IDAuMjUsIGtleS5wb3NpdGlvbiA9ICJ0b3AiLCBtYXguZXhwID0gMykKCiMgQ0Q0IG5haXZlIFQgY2VsbHMKRmVhdHVyZUhlYXRtYXAoaW1tdW5lLmNvbWJpbmVkLCBmZWF0dXJlcy5wbG90ID0gcm93bmFtZXMoQ0Q0bmFpdmVULmludGVyZmVyb24ucmVzcG9uc2UpWzE6OF0sIGdyb3VwLmJ5ID0gInN0aW0iLCBwdC5zaXplID0gMC4yNSwga2V5LnBvc2l0aW9uID0gInRvcCIsIG1heC5leHAgPSAzKQoKIyBCIGNlbGxzCkZlYXR1cmVIZWF0bWFwKGltbXVuZS5jb21iaW5lZCwgZmVhdHVyZXMucGxvdCA9IHJvd25hbWVzKGIuaW50ZXJmZXJvbi5yZXNwb25zZSlbMTo4XSwgZ3JvdXAuYnkgPSAic3RpbSIsIHB0LnNpemUgPSAwLjI1LCBrZXkucG9zaXRpb24gPSAidG9wIiwgbWF4LmV4cCA9IDMpCgpgYGAKCiMgU2F2ZSB0aGUgb2JqZWN0cwoKYGBge3J9CiMgc2F2ZSB0aGlzIHdvcmsgYW5kIGNvbWUgYmFjayB0byBpdCBsYXRlcgoKc2F2ZVJEUyhwYm1jLCBmaWxlID0gcGFzdGUoZGlyZWN0b3J5LCAiL3BibWNfY29tYmluZWQucmRzIiwgc2VwID0gIiIpKQpzYXZlUkRTKHN0aW0sIGZpbGUgPSBwYXN0ZShkaXJlY3RvcnksICIvc3RpbV9jb21iaW5lZC5yZHMiLCBzZXAgPSAiIikpCnNhdmVSRFMoaW1tdW5lLmNvbWJpbmVkLCBmaWxlID0gcGFzdGUoZGlyZWN0b3J5LCAiL2ltbXVuZV9jb21iaW5lZC5yZHMiLCBzZXAgPSAiIikpCmBgYAoKCg==